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INTRODUCTION 



THE PROBLEM 

What constitutes quality education? Even more important, 
how can quality education, how can teaching, and how can learning 
be improved? Quality education is defined here as teaching which 
results in optimum student achievement and student attitude toward 
subject. 



The major assumptions of this study were: 



1. Teaching and learning require effective inter- 
action between teacher and students. 

2. Quality education greatly depends on teacher 
performance in the classroom. 

3. Teacher effectiveness may be improved through 
organized feedback to teachers of observer 
ratings of the teaching process. 

The above assumptions were the bases for the major 

hypothesis of this study. This hypothesis was that by increasing 

and systematizing feedback to teachers, the teaching process will 

be improved with the result that student learning will increase. 

Accordingly, the following paradigm was tested: 

« 
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This study was. the second year (47) of an attempt to 
determine which of three types of feedback to teachers might 
result in a significant difference in student achievement and 
attitude toward courses of study. 



OBJECTIVES 



The general objectives were to determine if student 
achievement and student attitude toward school subjects: 

1. Can be improved significantly by systematically 
increasing feedback to teachers concerning pupil 
reaction to their teaching. 

2. Can be improved significantly by systematically 
increasing feedback to teachers concerning 
reaction of trained observers to their teaching. 

3. Correlate significantly with attitude of 
teachers toward their pupils. 

4. Can be improved to a greater degree by face- 
to-face feedback to teachers than by standard- 
ized feedback via mail. 



RELATED RESEARCH 

Kinhart (48) found that pupils of teachers who received 
ten hours of supervision did significantly better school work 
than pupils whose teachers were without supervision. Ten hours 
per month per teacher is costly. A need exists to determine if 
significant improvement in student growth can be achieved with 
the minimum hours of teacher supervision. 

Costs of improving the quality of education could be 
reduced even further if significant improvement in student growth 
could be effected through pupil teacher rating and feedback of 
results to teachers. Gage, Chat ter jee and Runkel (45) found that 
sixth-grade teachers will modify their teaching in the light of 
pupil rating if feedback of these ratings is given to the teachers. 
If feedback of pupil reaction to the teaching of their teachers 
has as beneficial a result as feedback of observer ratings, then 
a considerable savings could be made in both time and money. 

While a second rating after an appropriate interval 
subsequent to feedback may reflect change in the teaching process, 
a primary criterion should be end-product student measurement 
versus pre-measurement. As Bloom (3) insists, "The research 
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worker who wishes to understand teaching and teachers must under- 
stand not only the teaching and education process as it takes 
place, but also the outcomes or effects of the process - the 
changes that take place in the learners..." 

The investigator also felt the need to attempt to meas- 
ure the attitude of teachers toward their pupils. Does rating 
feedback affect the way teachers feel toward pupils? Ryans (17) 
found teachers, judged by their principals to be superior in 
teaching performance, held significantly and distinctly more 
favorable titudes toward their pupils than did teachers judged 
by their principals to be unsatisfactory or poor. Do classes 
which make the greatest gain have teachers whose attitude toward 
them as pupils is most favorable? The answers to these questions 
are needed to clarify the answer to the question of how to improve 
the quality of education. 

The proposed study was in keeping with the recommenda- 
tion of Ryans that studies be made "...of the influence of differ- 
ent in-service experiences of teachers..." (17) It was an attempt 
to silence criticism that educational research rarely seeks to 
improve the structure of what goes on in the classroom. (13) This 
study followed the advice of those who say: "...it is with 

respect to the teaching process itself that the greatest potential- 
ity for research lies..." (36) It was a study aimed at "doing 
something to change conditions rather than merely measuring and 
correlating them." (28) 
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METHOD 



GENERAL DESIGN 

Teacher evaluation and feedback to teachers were used in 
an attempt to improve the teaching process in order to improve 
pupil growth (see paradigm on page one) . The independent variables 
were the various types of feedback to teachers. The dependent 
criterion variables were: (1) end-of-year ratings and observations 

of teachers, (2) end-of-year teacher attitudes toward pupils, (3) 
end-of-year pupil achievement on standardized tests, and (4) end- 
of-year pupil attitude toward school subjects* 

"Feedback models have rarely been used in educational 
research." ( 25 ) Yet Flanders and others have found that it is 
possible to systematically record teacher-pupil classroom inter- 
action and thereby "assess the degree to which a teacher is 
meeting the social-emotional, as well as the intellectual needs 
of the pupils." ( 4 ) 

Design 4 (Pre-test - Post-test Control Group Design) as 
outlined by Campbell and Stanley ( 11 ) was adapted to this study. 
This design makes provision for sources of internal invalidity 
such as history, maturation, testing, instrumentation, regression, 
selection, mortality, and interaction of selection and maturation 
factors. The weaknesses of this design in controlling sources of 
external invalidity were counteracted to some extent by the way 
the sample was drawn and the manner in which the data was ana- 
lyzed. The design of this study may be diagrammed as follows : 
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In the above, R indicates random assignment to treat- 
ments. Oi, O3, O5, and O7 indicate pre-tests for achievement and 
attitude. X^, X2, X3, and X4 represent the treatment conditions 
while C>2> O4, 05, and Og were the post- tests for achievement and 
attitude. 



POPULATION AND SAMPLE 

The typical Pennsylvania elementary school is fairly 
typical of the typical American public elementary school. 
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In 1962-63, the average American school system spent $536 per 
pupil in average daily attendance, while the average school system 
in Pennsylvania spent $529 per pupil in average daily attendance. 
The average salary of the elementary teacher is $5,560 in the 
United States, while the same average in Pennsylvania is $5,610. 

In America the average school building is about 24 years old, while 
in Pennsylvania this average is about 30 years old. The average 
per capita personal income in 1962 for the United States was 
$2,357, while $2,368 was the case for Pennsylvania. (42) In many 
respects the typical Pennsylvania elementary school represents the 
typical American elementary school. 

The 851 elementary school systems of Pennsylvania were 
asked if they wished to participate in an experiment to improve 
teaching and learning. These schools were sent a summary of the 
proposed study. The schools which gave favorable replies were 
spread over a wide ’range of size and per-pupil expenditure. From 
this group of volunteering schools, a stratified random sample 
was selected of school systems involving 48 school buildings. 
Stratification was by per-pupil expenditure tb obtain a truly 
representative sample of the total population. 

Stratification of the sample by per-pupil expenditure 
hopefully accounted for other factors known to correlate with 
achievement and aptitude. Project Talent (43), which involved 

440.000 high school students, representative of the entire country, 
found that per pupil expenditure correlated .537 with family 
income, .535 with starting salaries of male teachers, .559 with 
starting salaries of female teachers, .307 with years of school- 
ing completed by parents and from .258 to .454 for seventeen 
achievement and aptitude areas. Also, it is noted that in the 

New York State Quality Measurement Project (27), involving over 

70.000 pupils, the good schools (based on student achievement) 
spent 25 percent more per pupil for instructional purposes than 
did the poor schools. 

To help limit the threat to external invalidity known 
as reactive arrangement, this study randomly selected entire 
, classrooms as normally scheduled units rather than randomly 
assigning students to treatments. The sampling representative- 
ness was increased by reducing M ...the number of students or 
classrooms participating from a given school... and (increasing) 
the number of schools in which the experiment was carried on. (11) 
Classrooms were also stratified by school subjects so that within 
each treatment an equal number of the following school subjects 
were represented; English, mathematics , social studies, reading 

and science. 

This study was limited to the sixth grade in order to 
simplify the problems of testing, control, and coordination. The 
pupils spread widely over the range of IQ and achievement except 
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no special education classes for children of limited ability or 
unusual handicaps were included. Eighty teachers were selected 
for participation. These teachers were assigned randomly to one 
of four treatments so that 20 teachers and their classes were in 
each treatment. There were also eight additional teachers and 
classes involved to guard against teacher losses due to sickness, 
death, pregnancy, etc. These eight additional teachers did not 
know they were alternates since they were involved in the same 
ways as were the other teachers. 

There were about 600 pupils per treatment or a total of 
approximately 2400 pupils in the entire study. Schools were 
selected from adjacent counties for efficiency of supervision and 
coordination. 



TREATMENT 1 

Treatment 1 involved pupil- teacher rating. Teachers 
need to see themselves as their students see them. The crucial 
test of teaching is how it changes the students who are being 
taught. Students are the only group who see their teachers day 
after day in the classroom. Students are not experts on the 
teaching process, but they can furnish valuable evidence which 
should be used to improve teaching. Objective teachers want to 
know how they can get along more successfully with their students. 

There is evidence that students are honest, reliable 
raters of teachers and that students can furnish valuable evidence 
even though they are not experts on teaching. Bryan (5) cites 
studies of student ratings with reliability coefficients ranging 
from .64 to .99 with an increase in reliability as the studies 
move from the college level to the intermediate elementary level. 

Leeds (33) found that 100 teacher-pupil inventory scores 
correlate as follows: 

.43 with ratings by their principals 

.49 with ratings by Leeds 

.45 with ratings by their students. 

Evidence of validity of pupil ratings is that such 
ratings have correlated as follows with principals’ ratings for 
the same teachers: 

.60 with discipline ratings 

. 70 with teacher-pupil relations 

.40 with ability to encourage learning. (40) 
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Starrak discovered that the opinions of students and of 
three faculty experts agreed seventy-five percent of the time. (38) 
Druckers and Remmers found that alumni rate their former teachers 
similarly to when as students they had rated the same teachers. 
(26). 



At Brooklyn College no appreciable differences in 
instructors’ scores were found to exist because of course grades 
received by students (6,681), size of classes, sex of students, 
college year, or whether or not the course was elective. (29) 
Remmers (37) and Hudelson (32) report low correlation coefficients 
(.07 and .19 respectively) between student ratings of college 
instructors and student grades received from their instructors. 

It appears that students, if approached properly, are honest, 
reliable raters of their teachers. 

Remmers also says: "If 25 or more student ratings are 

averaged, they are as reliable as the better educational and 
mental tests at present available." (15) In this experiment the 
teachers were given their average ratings by Item and for total 
since a few students can be expected to be much too far to one 
extreme or another. 

Gage, Runkel, and Chatterjee found that when sixth-grade 
teachers were furnished information on how their pupils described 
their actual teacher and how they described their ideal teacher, 
teachers' behaviors changed in the direction of their pupils' 
conception of an ideal teacher by the second time the pupils 
rated their actual teacher a month or two later. The feedback 
not only changed teacher behavior, but the teachers were able to 
more accurately predict their pupils' description of their teach- 
ing. 



Gage (45) and his associates found statistically sig- 
nificant differences for post-results between the experimental 
and control groups of teachers at the .05 level for only four of 
the 12 items on the pupil-teacher rating instrument. Also, the 
teachers who approached most closely to their pupils' ideal 
teacher were in the group with the longest, or about 53 days, 
between feedback and post-ratings of teachers. 

In Treatment 1 students rated their teachers every three 
weeks between September 26 and December 22, 1966. Feedback to 
teachers included class frequencies and averages for each item on 
the pupil-teacher reaction scale without any indication of how 
individual students rated them. In March and April the students 
again rated their teachers to see if teachers had improved in the 
eyes of their students. An interval of two and one-half months 
existed between fall feedback and spring rating of teachers. 



TREATMENT 2 



Treatment 2 involved a systematic recording (by two 
carefully trained observers) of classroom teacher-pupil inter- 
action four times in the fall (between September 26 and 
December 22, 1966) and twice in the spring (between March 6 and 
April 25, 1967). An interval of two and one— half months existed 
between the fall feedback and spring observation of teachers. 

Research on improvement of teaching requires observation 
of teachers while they teach. Medley and Mitzell note: "Certainly 

there is no more obvious approach to research on teaching than 
direct observation of the behavior of teachers while they teach 
and pupils while they learn. Yet it is a rare study indeed that 
includes any formal observation at all." (14) 

Ryans offers this pertinent advice: 

Of the measurement approaches employing 
observation and assessment of teacher 
behavior in process only time sampling 
involving replicated systematic obser- 
vation by trained observers produces 
sufficiently reliable data to recommend 
its use in fundamental research. . . (16) 

Medley and Mitzell point out that validity of measure- 
ment of behavior depends on: (1) a representative sample; (2) 

accurate recording; and (3) scoring which faithfully reflects 
differences in behavior. (14) 

The two observers used in this study were trained in 
interaction analysis in the first week of September 1965 at 
Temple University and they had performed interaction analysis 
some 1680 times during the previous year in a similar study on 
the secondary level. A high degree of reliability (.90) was 
obtained between these two observers. The observers were randomly 
assigned to observe teachers. Each observer observed one-half of 
the teachers in each treatment and observed the same randomly 
selected teachers throughout the study. 



TREATMENT 3 

In this treatment teachers received feedback of both 
student ratings (as in treatment 1) and observations based on 
classroom interaction analysis (as in treatment 2) . Both of these 
kinds of feedback were based on two occasions in the fall semester 
(September and October) and once in the spring semester (March) 
with a 2Qr>week interval between last fall feedback and the first 
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spring observation. 



TREATMENT 4 

In this treatment the teachers received only the pre- 
test results for pupil achievement and attitude toward subjects 
(English, arithmetic, science, reading and social studies). 



ADDITIONAL TREATMENT PROCEDURES 

In treatments 1, 2, and 3 all teachers received the 
same pre-test results as did the teachers in treatment 4. In 
treatments 1 and 4 teachers were observed without feedback twice 
in the fall and once in the spring. In treatments 2 and 4 stu- 
dents rated their teachers twice in the fall and once in the 
spring without feedback to the teachers. 

One-half of the teachers in each treatment were selected 
randomly for face-to-face feedback while the other half received 
standardized mailed feedback only. This resulted in a 2 by 4 
factorial design with 10 teachers and their classes per cell as 
indicated below: 



NUMBER OF 1 


TEACHERS 


BY TYPE OF 


FEEDBACK AND 


TREATMENT 




Type of Feedback 


x i 


TREAT 

x 2 


M E N T 

x 3 


x 4 


Totals 


Face-to-Face 


10 


10 


10 


10 


40 


Via Mail 


10 


10 


10 


10 


40 


Totals 


20 


• 20 


20 


20 


80 



IN-SERVICE EDUCATION 

During the second and third weeks of July 1965, the 
co- investigators visited with each involved chief school admin- 
istrator to explain the study and to obtain their cooperation. 

All involved teachers and administrators in all treatments and 
controls received two days of orientation on the 17th and 24th 
of September to explain the purpose and importance of the study. 

A climate of understanding and acceptance was the goal. Teachers 
also completed the Minnesota Teacher Attitude Inventory and worked 
a programmed learning approach to Flanders classroom interaction 
analysis (Appendix A). Dr. Edmund J. Amidon and Dr. Anita Simon 
of Temple University explained how teachers could use results of 
the Flanders system. On April 15, 1967 teachers and administrators 
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also participated in a spring "round-up". After teachers completed 
the Minnesota Teacher Attitude Inventory and an attitude question- 
naire concerning the study, they were briefed on results to date, 
their questions were answered by. the investigators, and the final 
testing procedures were explained. 



TEACHER RATING BY OBSERVERS 

Medley and Mitzell’s (14) review of research led them to 
conclude that teacher observation should be separated from quan- 
tified rating. Since Anderson's (22) pioneer work, others such 
as Bales, (2) Bush, (10) Thelen, (19) Withall, (41) Lewis, (34) 
Flanders, (44) and Amidon (21) have investigated student- teacher 
interaction as a determiner of effective teaching. 

In this study the Flanders system of teacher-pupil 
interaction analysis was used. Flanders and his associates use 
just ten .categories (see Appendix A) for recording teacher-pupil- 
verbal-response behavior in the classroom. Every three seconds 
the observer writes down the category number of the interaction 
he has just observed. These numbers are recorded in .sequence in 
a column and after the class are easily quantified into a matrix 
which provides insight into the teaching process. "With' the use 
of this recording procedure it is possible to assess the. degree 
to which a teacher is meeting the social-emotional, as well as 
the intellectual, needs of the pupils." (4) 

The validity of the Flanders technique is self-evident 
since it permits the recording of immediately perceived behaviors. 
The reliability of the procedure was assured through the use of 
observers thoroughly trained and experienced with interaction 
analysis. Satisfactory coefficients of observer reliability were 
obtained before the actual experiment commenced. 

The .Flanders instrument has been successfully field 
tested with 900 students, half in seventh-grade social studies 
and half in eighth-grade geometry. It was found that "...in both 
content areas the students of the more indirect teachers scored 
higher on achievement tests than did students of the more direct 
teachers." (1)’ It also found that the more flexible teachers who 
could shift readily from direct to indirect and vice versa ap- 
proaches had more success improving student achievement than did 
teachers with less flexibility in their classroom behavior. 



PUPIL RATING OF TEACHERS 

A great number of pupil-teacher rating instruments have 
been devised and are of value but' a more satisfactory instrument 
is needed... Barr has examined many investigations of teaching 



efficiency of the past several decades and he has found them pay- 
ing insufficient attention to "...The particulars of teaching..." • 
(23) and the relationship between teacher and student. (24) Those 
devising teacher-rating instruments seem to have ignored the 
uni dimensional scalogram technique. Yet McNemar conceded as early 
as 1946 that a scale devised by this technique has "...superiority 
on the single dimension problem..." (35) 

A uni dimensional scale places each item along a single 
continu um similar to the inches ‘of a yardstick. Each total score 
tells which items were reacted to favorably and which were reacted 
to unfavorably. Intensity analysis can be used to determine which 
score represents a dividing line between favorable and unfavorable 
attitudes. A satisfactory unidimensional scale is derived from 
the universe of attributes which define the concept and have a 
common content. 

Since there is evidence that the Hayes Pupil-Teacher 
Reaction Scale (Appendix B) may be uni dimensional, it was used in 
this study. Several administrations and successive refinements 
in 1960 of the Hayes Scale indicated that there are certain desir- 
able behaviors which are generally characteristic of good teachers 
and that these behaviors are not generally characteristic of poor 
teachers. (46) 

In 1961 a follow-up study (46) with the Hayes instrument 
resulted in what appears to be a uni dimensional instrument to 
measure attitude of students toward the teaching effectiveness of 
their teachers. 



TESTING PROGRAM 

On September 12, 1966 test coordinators from the 48 
elementary schools involved attended an orientation meeting at 
East Penn Union Junior High School, Emmaus, Pennsylvania, where 
adherence to standardized test directions and time limitations 
was stressed. During the third week of school these test coor- 
dinators had administered the following; the Remmers Scale to 
Measure Attitude Toward Any School Subject, and Form X of the 9 
edition of the Stanford Achievement Test, Intermediate II Battery. 
During the weeks of April 16 and 23, Form Y of the 1964 edition of 
the Stanford Achievement Test, Intermediate Battery II was admin- 
istered as a post-test. The Remmers Scale to Measure Attitude 
Toward Any School Subject was administered during April as another 

post-measurement . 



ANALYSIS OF DATA 

• Can student achievement be improved significantly by 
feedback to teachers ofr 
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1. Pupil reaction to their teaching? 

2. Reaction to their teaching hy trained observers? 

3. Results on pre-pupil growth evaluation? 

Since intact classes had been assigned randomly to treat- 
ments, the' class achievement means were used as the basic obser- 
vations and treatment effects were tested against variations in 
these means. Data was also analyzed with individual pupil scores 
as the basic observations to see if statistically significant 
results would occur. Where necessary, covariance was used in addi- 
tion to analysis of variance. Additional comparisons were made of 
achievement results for classes whose teachers are pr ima rily direct 
(commanding, lecturing, criticizing, justifying) versus classes whose 
teachers are primarily indirect (accepting, asking, praising, en- 
couraging) in their teaching. 

Can student attitude toward school subjects be improved 
significantly by feedback to teachers of: ' 

1. Pupil reaction to their, teaching? 

2. Reaction to their teaching by trained observers? 

i 

3. Results of pre-experimental pupil evaluation? 

.Analysis of variance and covariance ; if needed, were 
used to analyze the data. 



Does attitude of teachers toward their pupils correlate 
significantly with student achievement and student attitude toward 
school subjects? Pearson product moment correlation coefficients 
were computed between teacher inventory scores and class achieve- 
ment means and also between teacher inventory scores and attitude 
toward subjects means. 

A Pearson product moment correlation coefficient was 
computed between each set of pupil ratings of teachers to obtain 
a test-retest indication of reliability. Unidimensional Cornell' 
scalogram analysis was applied to pupil ratings of teachers. (30) 
Analysis of variance and covariance was used to compare results. 
Frequency tables were prepared to help interpret the results of 
the teacher opinion questionnaire (Appendix D) . 



TEACHER REACTIONS 

To obtain the reactions of teachers to the study, a 
questionnaire (Appendix C) was administered to all teachers. 



RESULTS - PART ONE 



TEACHER REACTIONS 

Eighty-eight percent of the 60 teachers receiving feed'" 
back (based on pupil ratings or classroom observation or both) 
reported that the feedback provided a critical analysis of their 
teaching and an objective basis for improvement. Only six of the 
40 teachers who received face-to-face feedback (in addition to 
written feedback) would have preferred written feedback only. 
Seventy-three percent of the 40 teachers who received written 
feedback only would have preferred to have been given face-to-face 
feedback also. 

Eighty-one percent of the 80 teachers said they adequately 
understood the purpose of the study. 'Forty percent thought too much 
was covered in the two orientation days, while forty-four percent 
felt instruction was not sufficient for them to accurately interpret 
classroom interaction analysis matrices. Yet* 90 percent of teachers 
receiving feedback on classroom interaction analysis indicated they 
had a meaningful basis for changing their teaching. Also, 90 percent 
of teachers who received feedback on their pupil ratings felt that 
most of their pupils accurately rated their teaching. Of the 40 
teachers receiving feedback on pupil ratings , 83 percent reported the 
benefits warranted widespread use of pupil rating of teachers. Also, 
86 percent of all 80 teachers agreed that the benefits derived from 
classroom interaction analysis were such that this should be used 
widely in teacher in-service training programs. A more detailed 
description of teacher reactions to the study may be found in 
Appendix D. 



RESULTS - PART TWO 

Since intact classes had been assigned randomly, the class 
means were used as the basic observations for analysis. 



ACHIEVEMENT MEANS 

* 

Student achievement was measured at the beginning and at 
the end of this study by Stanford Achievement Test complete battery, 
using form X for pre-testing and form Y for post-testing. The 
Stanford Achievement Tests included the following individual tests: 
Word Meaning, Paragraph Meaning, Spelling, Language, Arithmetic 
Computation, Arithmetic Concepts, Arithmetic Applications, Social 
Studies, and Science. Composite scores for Reading were obtained 
by combining results in Word Meaning and Paragraph Meaning. Compos- 
ite scores for Mathematics were obtained by combining results in 
Arithmetic Computation, Arithmetic Concepts, and Arithmetic 
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Applications. The Language Test was used for English Achievement 
while Social Studies and Science Achievement were measured by the 
Stanford Tests in these areas. Spelling was not used in the sta- 
tistical analysis since it was not considered as an independent 
course in this study. 

The pre- and post-class achievement means are shown in 
Appendix E, Tables 9 and 10. Analysis of variance (Appendix E, 
Tables 11 and 12) revealed no significant differences among either 
the pre- or post-achievement means. 



STUDENT ATTITUDE TOWARD SCHOOL SUBJECTS 

The pre- and post-test class attitude means were obtained 
from the Remmers Scale to Measure Attitude Toward Any School Subject 
(Appendix E, Tables 13 and 14). Analysis of variance indicated no 
significant differences in either the pre- or post-tests. 

(Appendix E, Tables 15 and 16). 



STUDENT RATINGS OF THEIR TEACHERS 

The class means, in raw score form, for the first and 
last cycle of student ratings are listed in Appendix E, Tables 17 
and 18. Statistical analysis of these means is reported in 
Appendix E, Tables 19 and 20. No significant differences were 
found. 



Cornell scalogram analysis also was performed on the 
Hayes Pupil-Teacher Reaction Scale (Appendix B) . The response 
categories of the Hayes scale were dichotomized for scoring pur- 
poses prior to applying scalogram analysis. The justification 
for combining categories for scoring purposes is that verbal 
habits of people differ. The meaning of "extremely clear" to 
some is the equivalent of the meaning of "very clear" to others. 
These two categories might be combined for scoring purposes and 
given the weight of one. Likewise, the meaning of "sometimes" 
might mean to some what "seldom" means to others. The preceding 
two categories might be combined and given a weight of zero. 

The rule followed to combine response categories was to 
draw a line between the responses so as to minimize the number of 
pupils in the low group above the line and number of pupils in the 
high group below the line (12). Item one was analyzed as follows; 
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Item 


Original 

Weights 


Low 25% 


High 25% 


This teacher makes the 
lesson clear in trie first 
few minutes of class. 

a Always 


3 


58 


412 


b Usually 


2 


280 


214 


c Sometimes 


1 


276 


13 


d Seldom or Never 


0 


27 


2 



Pupil responses were scored first using the above weights. 
Then, based on total scores, the lowest 25% of the pupils and the 
highest 25% were determined. Next, categories were combined so that 
responses "a" and "b" were recorded as ”1” and responses "c" and "d" 
were recorded as "0 M for this item. 

The same procedure was repeated for, all items and the 
following scoring key resulted: 



Item Responses Scored as 1 Responses Scored as 0 




a 

a, b 
a 

a, b 
a 

a, b 

a 



b, c, d, 
c, d 
b, c, d, 
c, d 
b, c, d, 
c, d 
b. c. d. 



The above key resulted in a maximum possible score of 
nine or a minimum score of zero in content. 



Cornell scalogram analysis was then performed as follows 

(28): 

1. The IBM cards were arranged into rank order by total scores from 
high to low. 

2. A table was prepared with: 

a. One row for each person’s score, using the rank order arrived 
at In step one above. 

b. One column for each category of each question. 

3. The responses of each person were indicated on the table by plac- 
ing a check (/) in the appropriate column for each item opposite 
the row representing the total score of that person. 
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4. Then the frequencies for each category were totaled. The sum of 
the frequencies in both categories for each question had to equal 
the total number of subjects answering the questionnaire. 

5. Cutting points were placed for each item at the place in the 
rank order where the most common response changed from a favorable 
category to a non-favorable one. Cutting points were established . 
to minimize error. 

6. The errors were totaled for each item using the cutting points to 
determine which responses fell outside the established patterns. 

7 . The coefficient of reproducibility for each item was computed by 
dividing the number of errors (for that item) by the total number 
of subjects and subtracting the result from one. 

8. The marginals (percent of people who answered each item favorably) 
were computed for each item. 

Appendix E, Tables, 21 and 22 show a^ sample of the coeffi- 
cients of reproducibility and marginals resulting from scalogram 
analysis of the first two ratings in the fall. 

# 

The marginals (or percentage of the items answered favor- 
ably) ranged from .24 to .79 with an average .49 marginal. Item 
one had a marginal of .34 which meant that the coefficient of re- 
producibility could not be less than .66 for this itemi The coef- 
ficient of reproducibility was .83 for this item. 

« 

Also the marginal for item seven was .63. This meant that , 
the coefficient of reproducibility for this item could not fall below 
.63. A coefficient of reproducibility of .78 was obtained for this 
item. This represents a gain of .15 over the minimum obtainable 
reproducibility. The average minimum marginal reproducibility for 
all nine items was .69 compared to the .80 average coefficient of 
reproducibility . 

When intensity analysis was performed, a point was found 
which divided the sample into those with favorable and those with 
unfavorable attitudes. This point was zero for both the spring and 
fall intensity analyses, indicating any score but zero was a favorable 
attitude on the part of the students toward their teachers* effective- 
ness (Figures 1 and 2 on the following pages). 
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MEDIAN INTENSITY PERCENTILE 



FIGURE 1 

INTENSITY CURVE FOR CYCLES 1, 2, 3 AND 4 
HAYES PUPIL-TEACHER REACTION SCALE 




MIDPOINT OF CONTENT PERCENTILE 
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MEDIAN INTENSI 



FIGURE 2 

INTENSITY CURVE FOR CYCLES 5 AND 6 
HAYES PUPIL-TEACHER REACTION SCALE 
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The foldover technique was used to determine intensity 
scoring. The answers to' each item were dichotomized. An item 
answered with "a" or "d" was scored as "1", and an item answered . 
w ith "b" or "c" was scored as ”0”. The rationale for this techni- 
que is that a student answering an item with "a" or "d" feels more 
intensely favorable or unfavorable toward an item than one who 
answers with "b" or "c". This technique meant that intensity 
scores ranged from zero tu nine. 

the intensity and content scores (based on the number of 
favorable replies) were then recorded in a matrix. The cumulative 
percentiles were computed for content scores and then the midpoints 
of content percentiles were computed. Next, the cumulative percen- 
tiles for intensity scores were computed and the median intensity 
for each content score was computed. The median intensity for each 
content score was then plotted against the corresponding midpoint 
for content percentile. 

Appendix E, Table 23 contains the intensity table for a 
combination of the fall visitation cycles, and Table 24 contains 
the intensity scores for the spring cycles. 



INTERACTION ANALYSIS 

The large I/D (ratio of indirect to direct teacher talk) 
and revised i/d (motivation versus control) fall and spring ratios 
were investigated by analysis of variance to determine the change 
in interaction as a result of the feedback process. Appendix E, 
Tables 25 and 26 contain the ratios used in this investigation. 

No significant differences materialized for either I/D 
or the revised i/d. Appendix E, Tables 27, 28, 29, and 30 report 
the results of this analysis. 

Analysis of variance, two-way classification by type of 
feedback and treatment, failed to produce any statistically signif- 
icant results when applied to measures taken from each teacher s 
matrix. These measures were: extended indirect, extended student 

talk, I/D for rows 8 and 9, revised i/d for rows 8 and 9 and the 

3,3 cell. 

Interaction analysis results, giving the range of scores 
and percentages of time teachers spent in each of the ten categories 
of Flanders’ matrix, are shown in Appendix E, Tables 31 and 32. 



MINNESOTA TEACHERS ATTITUDE INVENTORY (MTAI) 

The pre- and post-test scores of the MTAI (Appendix E, 
Tables '34 and 35) were examined by analysis of variance, two-way 
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classification by treatments and feedback. 

The post-test analysis did not yield any significant 
results in the treatments or feedback. Appendix E, Tables 34, 35 
and 36 comprise the findings of these analyses. 



CORRELATION COEFFICIENTS 

Coefficients of consistency for results on various ad- 
ministrations of the Hayes Pupil-Teacher Reaction Scale range from 
.58 to .85 (Appendix E, Table 37). Additional correlations are 
contained in Appendix E, Table 38. 

Student achievement (Stanford) and teacher attitude 
toward pupils (MTAI) had correlations of -.16 at the beginning of 
the year and -.004 at the end of the school year. Student attitude 
toward school subjects (Remmers Scale) and student achievement 
(Stanford) correlated -.27 at the beginning of the year and -.33 
at the end of the school year. 

The correlation between student attitude toward teaching 
of teachers as measured by the Hayes Pupil-Teacher Reaction Scale 
and student attitude toward school subjects (Remmers) was .37 at 
the beginning of the year and .45 at the end of the school year. 
Negative correlations were obtained between teacher attitude (MTAI) 
and student attitude toward school subjects (Remmers). 

End of year principals' ratings of their teachers corre- 
lated .59 with teacher attitude toward pupils as measured in 
April 1967 by the MTAI. Principals' ratings also correlated at 
.27 with student attitude toward teaching effectiveness as measured 
in April by the Hayes Pupil-Teacher Reaction Scale. 



RESULTS - PART THREE 



The analysis of data in part two of the results were 
based on class means of intact classes because of the necessity to 
randomize the sample in this manner. 

The investigators also decided to analyze certain data 
further using the results for individual students. Since students 
were selected for their respective classes by local administrate 
several months before classes were randomly assigned to treatmen s, 
there was no bias on the part of the investigators in student 

selection. 



ACHIEVEMENT 

The individual results of the Stanford pre- and post- test 
achievement scores were subjected to analysis of variance two-way 
classification (Appendix E, Tables 39 and 40). . 

The analysis on pre-test scores revealed treatments to be 
significant at the .10 level and post-test scores significant i 
both treatments and type of feedback at the .10 level. 

, ± Ti ' e ^t he llllllsTllsTs S V SfcovariaL a (Applndix 

E f Table 41)! The Significant level of treatments remained at the 
io level However, S the type of feedback and interaction in the ■ 
covariance analysis were significant at t e ev . 

The adjusted means for type of feedback we ^ e ^^her 
analyzed by Winer's F Test for multiple comparison of means. ( ) 

It was found that .£ » n f ZZllZ two 

"Yl fev flTd Srea m^thrSe^ Significantly different 
from^ treatment 6 two at the .05 level. These differences are shown 

as follows: 



Face- to-Face Feedback • 

Differences 




4 

3 

1 

2 



64.08 

63.11 

62.93 

61.77 



.97 



1.15 

.18 



2.31** 

1.34* 

1.16 



** Significant at .01 level 
* Significant at .05 level 
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In the analysis of written feedback only, it was found 
that achievement in treatment one was higher than in treatments two, 
three and four at the .01 level of significance. This can be seen 
in the following: 



Written Feedback 



Treatment 


Means 


Differences 
12 3 


4 


1 


66.83 


2.01** 


2.69** 


3.00** 


2 


64.82 




.68 


.99 


3 


64.14 






.31 


4 


63.83 








** Significant at 


.01 level 






• 



When the adjusted means for the written feedback only 
were compared to the face-to-face feedback adjusted means, signifi- 
cant differences at the .01 level were found as indicated below by 
the double asterisk: 

Written Feedback 




Treatment 




1 


2 


3 


4 








66.83 


64.82 


64.14 


63/83 




« 


Means 




Differences 






4 


64.08 


2.75** 


.74 


.06 


.25 


Face-to-Face 


3 


63.11 


3.72** 


1.71** 


1.03 


.72 


Feedback 


1 


62.93 


3.90** 


1.89** 


1.21 


.90 




2 


61.77 


5.06** 


3.05** 


2.37** 


2.06** 



** Significant at .01 level 



STUDENT ATTITUDE TOWARD SCHOOL SUBJECTS 

Student attitude toward school subjects was investigated 
in fall and spring by analysis of variance two-way classification, 
by treatment and type of feedback (Appendix E, Tables 42 and 43). 

The fall analysis was significant at the .05 level for 
treatments only. The spring analysis detected no significant 
differences in student attitude. 

No significant differences were found in the analysis of 
covariance of post-test scores using the pre-test score as the co- 
variate (Appendix E, Table 44) » 
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STUDENT RATINGS OF THEIR- TEACHERS 

Individual student ratings of their teachers were investi- 
gated in the fall and spring by analysis of variance (Appendix E, 
Tables 45 and 46) . 

There were no significant differences found in the fall 
analysis. However, significant differences in the spring analysis 
were disclosed in type of feedback and interaction at the .01 level. 

The student ratings of their teachers in the spring were 
examined by analysis of covariance using the student ratings in the 
fall as the covariate (Appendix E, Table 47). 

The analysis of covariance revealed ' significant differ- 
ences in student ratings at the .05 level for treatments, while 
differences in both type of feedback and interaction were signifi- 
cant at the .01 level. 

The adjusted means were analyzed by Winer’s method of 
multiple range testing. (20) 

In the analysis of adjusted means for treatments, treat- 
ment three and treatment one were found to be significantly differ- 
ent than treatment four at the .05 level. The differences were as 

follows : 




* Significant at .05 level 

In the examination of adjusted means for face-to-face 
feedback, the difference between treatment three and treatment four 
was significant at the .01 level. In addition, treatment three was 
significantly different from treatment one, and treatment two also 
was significantly different from treatments one and four at the 

. 05 level. 
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Face~to-Face Feedback Means Differences 



Treatment 


Means 




Differences 




3 


2 1 


4 


3 


4.54 




.02 .31* 


.36** 


2 


4.52 




.29* 


.34* 


1 


4.23 






.05 


4 


4.18 









** Significant at .01 level 
* Significant at .05 level 

The investigation of written feedback only revealed sig- 
nificant differences at the .01 level between treatments one and 
four, treatments one and two, and treatments three and two. Sig- 
nificant differences at the .05 level were shown to be between 
treatment one and treatment three, and between treatment three and 
treatment four. This can be seen in the following summary: 



Written Feedback Means Differences 



Treatment • 


Means 




Differences 




'l 


3 4 


2 


1 


5.16 




.27* .54** 


.76** 


3 


4.89 




.27* 


.49** 


4 


4.62 






.22 


2 


4.40 









** Significant at .01 level 
* Significant at .05 level 

When the adjusted means for the written feedback only were 
compared to the face-to-face feedback adjusted means, significant 
differences at the .01 level were found as indicated below by the 
double asterisk: 



Means Differences 


Between 


Written 


Feedback 




and Face- 


-to-Face 


Feedback 






Treatment 




1 


3 


4 


2 






5.16 


4.89 


4.62 


4.40 




Means 


Differences 




3 


4.54 


.62** 


.35** 


.08 


.14 


Face-to-Face 2 


4.52 


.64** 


.37** 


.10 


.12 


Feedback 1 


4.23 


.93** 


.66** 


.39** 


.17 


4 


4.18 


.98** 


.71** 


.44** 


.22 



** Significant at .01 level 
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DISCUSSION 



The consistency of student attitude toward teaching j f 
teachers as measured by the Hayes Pupil-Teacher Reaction Scale 
tended to support previous research that students are reliable 
raters. The correlation between student rating of teachers and 
principal’s ratings of teachers indicated reasonable validity for 
student ratings. Cornell scalogram analysis produced an average 
coefficient of reproducibility of .82 for the items on the Hayes 
Scale. Intensity analysis indicated that an attitude score greater 
than zero was a favorable attitude toward the teaching of teachers 
as measured by the Hayes Scale. The average percent pro (favorable) 
was .50 for this study.' The items on the Hayes Scale could be 
consistently rank ordered along a continuum with the item marginals 
or favorable response proportions ranging from .24 to .79. A 
significant correlation (.45 on class means) was obtained between 
student ratings of their teachers’ teaching and student attitude to 
toward school subjects. 

The conclusions concerning achievement and attitudes vary 
somewhat depending on whether or not one is willing to accept only 
results based on statistical analysis when class means are used as 
the basic observations. The conservative answer to this question 
is to use class means for statistical analysis since intact classes 
were rando ml y assigned to treatments. On the other hand, a case 
could be made for analysis of individual scores, since assignment 
of students to classes by school administrators was done several 
months prior to random assignment of intact classes by the investi~ 
gators who were from outside the involved school systems. 



CONCLUSIONS 



In this study when class means were used to analyze the 
data, student achievement and student attitude toward school sub- 
jects was not improved significantly by systematically providing 
feedback to teachers of (1) pupil reaction to the teaching of 
their teachers, or (2) results of classroom interaction analysis 
using Flanders' procedures, or (3) a combination of pupil reac- 
tions and classroom interaction analysis. As measured in this 
study neither student achievement nor student attitude toward 
school subjects were significantly correlated with attitudes of 
teachers toward their pupils. Also, when class means were used 
to analyze the data, there were no significant achievement or 
attitudinal differences between face-to-face feedback and stand- 
ardized feedback via mail. 

When individual scores were analyzed, significant re- 
sults in achievement were obtained in favor of written feedback 
only over face-to-face plus written feedback. Also, when indi- 
vidual student ratings of the teaching of teachers were analyzed, 
significant differences resulted in favor of: (1) written feed- 

back only and (2) treatments 1 and 3 versus 4. (It is noted that 
a common element in treatments 1 and 3 was feedback to teachers of 
results of student rating, whereas teachers in treatment 4 re- 
ceived only pre-test results for pupil achievement and attitude 
toward school courses.) Analysis of individual scores in classes 
whose teachers received written feedback only, indicated signifi- 
cant differences favoring treatment 1 over each of the other three 
treatments in both student achievement and student ratings of 
teachers. 



Most teachers felt that the benefits derived from both 
pupil ratings and classroom interaction analysis warranted their 
wide use in teacher in-service programs. 



IMPLICATIONS 



It appears necessary to provide frequent and intensive 
help to teachers in order to change teacher behavior and to im- 
prove teacher effectiveness. The teacher questionnaire results 
indicated that both the Flanders classroom interaction analysis 
and Hayes Pupil-Teacher Reaction Scale may be useful instruments 
in providing insight to teachers as to how they can improve their 
teaching. It should be noted that the Hayes Scale takes only a 
few minutes to administer and to analyze and yet provide a re- 
liable, reasonably valid way to help teachers improve their teach- 
ing. 



SUMMARY 



The purpose of this study was to determine if student 
achievement and student attitude toward school subjects in the 
sixth grade: 

1. Can be improved significantly by systematically in- 
creasing feedback to teachers concerning pupil 
reaction to their teaching. 

2. Can be improved significantly by systematically 
increasing feedback to teachers concerning reaction 
of trained observers to their teaching. 

3. Correlate significantly with attitude of teachers 
toward their pupils. 

4. Can be improved to a greater degree by face-to-face 
feedback to teachers than by standardized feedback 
via mail. 

In treatment 1, students rated their teachers four times 
in the fall. After these ratings, half the teachers were mailed 
feedback and the other half received face-to-face feedback as well 
as written feedback. This feedback included class frequencies and 
averages for each item on the pupil-teacher reaction scale. In 
the spring, the students rated their teachers twice. There was an 
interval of two and one-half months between the last fall and the 
first spring rating. 

Treatment 2 involved systematic recording of classroom 
teacher-pupil interaction (by two carefully trained observers) 
four times in the fall and twice in the spring. An interval of 
about four months existed between the two types of feedback in the 
fall and the first spring observation. 

In treatment 3, teachers received feedback of both 
student ratings (as in treatment 1) and observations based on 
classroom interaction analysis (as in treatment 2) . Both kinds 
of feedback were based on two visitations in the fall semester 
and once in the spring semester. There was a 20-week interval 
between the fall and spring feedback. 

In treatment 4, although teachers were rated by their 
students and observed twice in the fall and once in the spring, 
they received feedback only on the pre-test results for pupil 
achievement and attitude toward subjects. Teachers in treatments 
1, 2, and 3 also received the same kinds of information provided 
teachers in treatment 4. 
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In the analysis, of the data using class means, no sig- 
nificant differences were found in achievement or attitudinal means. 

When individual scores were analyzed, significant differ- 
ences were found in the student ratings of the teaching of teachers. 
The preceding results favored treatments 1 and 3 over treatments 2 
and 4 at statistically significant levels. Also, when individual 
achievement scores and student ratings were analyzed, written 
feedback tended to be more effective than face-to-face feedback. 
Analysis of individual scores in classes whose teachers received 
written feedback only, indicated significant differences favoring 
treatment 1 over each of the other three treatments in both student 
achievement and student ratings of teachers. 
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APPENDIX A 



INTERACTION ANALYSIS: A PROGRAM FOR INTERPRETATION OF MATRICES 



Directions : 



See "Categories for Interaction Analysis" on last 
page and then return to Phase I below. 



PHASE I - INTRODUCTION TO MATRIX 

The observer becomes familiar with the classroom 
atmosphere before beginning to record interaction. Once 
the observer begins- recording numbers, he categorizes the 
classroom interaction every three seconds or every time a 
change in categories occurs. After the observer ceases 
recording, he has a list of category numbers which is the 
raw interaction data. In order to interpret this data, he 
can produce a systematic summarization by entering the 
category numbers into a matrix. The matrix preserves the 
sequence pattern of events, but does not preserve the 
temporal order of events. 



1-1 The matrix could best be described as preserving: 

(a) Temporal order of events 

(b) Quantified sequence of events 

(c) Nonverbal behavior 



(b) 



This is a Matrix. 




o 



A-l 
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The matrix consists of 10 rows and 10 columns. The rows 
run horizontally and the columns run vertically. Each unit in the 
matrix is called a cell. 




row 



column 



1-2 A matrix consists of 10 



and 10 



1-3 Each unit in the matrix is called a 



1-4 The rows run 
run 



and the columns 




1-4 

cell 



rows and 
columns 
,. cell 

horizontally 

and 

vertically 



Filling in a matrix is called tabulating 
or tallying. After he has recorded his raw data 
(See A) in the classroom, the observer begins to 
tabulate the raw data into the matrix. To tabulate 
into a matrix, the observer must use the first 
number he has recorded as the row number, and the 
second number as the column number. For example, 
if the first number is "1" and the second number 
is ”2”, enter a tally mark into the matrix in row 
one, column two (See B) . It is important to note 
that this pair of numbers is recorded as one tally. 



A 

Row 4 

Column ^2 

3 

4 
8 
8 
2 
3 

1-5 Filling in a 



B 

Column 

Z 



T“1 

Row ^ - - jL. u i n- | » | 

l-l-LITI 

matrix may be described as 



tabulating or 
tallying 
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1-6 In a consecutive pair of numbers, the first is called 
the number. 

1-7 The second number is called the ; number. 

1-8 Each Tally in the matrix represents recorded 

numbers . 

1-9 If a consecutive pair of numbers is 2 and 4, then 

the row number is and the column number is . 

1-10 The cell in which the above pair of numbers is placed 
is called the cell. 



In the example of a segment of classroom inter- 
action (See C) we notice that each number is used twice. 

C 

1* Row 
2 ' Column 
'3. Row 
A' Column 
Row 

8' Column 

<2 

1-11 The column number of the first pair becomes the 

number of the second pair. 

1-12 Using the same procedure, we can see that the third 

pair of numbers has as its row number the number 

and as its column number the number • 

1-13 The fourth pair of numbers has as its row number the 
number and as its column number the number . 

1-14 The tabulation or tally for the fourth pair of numbers 
would therefore be placed in the cell. 



Row 

Column 

Row 

Column 

Row 

Column 



PHASE II - APPLICATION OF RAW DATA TO MATRIX 
(Refer to Figure 1) 

We shall now construct a matrix from a summari- 
zation of data obtained from a 15-minute class period. 



A- 3 



row 

column 
two 
2, 4 
2,4 



row 

3 

4 

4, 8 
4,8 



0 
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Example: Teacher: "John, What day of the week is it?" 

John: "Today is Friday." 

This dialogue would be categorized as a 4,8, and one 
tally or tabulation would be placed in the 4,8 cell. 

2-1 The 4,8 cell is located at the intersection of 
4 and 8. 

2-2 If there were 16 such interactions during the 

class period, we would place the number 

in the 4,8 cell. 



Directions: Insert the given number of tallies in 

their proper cells to complete the 
practice matrix. (Figure 1) 

2-3 The teacher has interrupted his lectures with ques- 
tions 9 times. The 9 would be placed in row , 

column • Place 9 tallies in the 5,4 cell. 

2-4 The teacher has continued to lecture after asking 
a question 4 times. These tallies belong in 
£ow t column . Place 4 tallies in the 

proper cell. 

2-5 The teacher has asked extended or long questions 11 

times. These tallies go in row , column . 

Enter the tallies in the -proper cell. 

2-6 The students have responded to direct questions with 
a narrow, factual answer 44 times. You can summarize 
this particular interaction by placing the number 

in row 4, column 8. Enter the tallies in the 

proper cellv 

« 

2-7 There were no responses to teacher questions in 

which the students presented their own ideas ; there- 
fore, there are tallies in cell 4,9. 

2-8 A factual student response was followed 29 times by 
teacher praise or encouragement. The number 29 goes 
into the cell. Enter the tallies in the 

proper cell. 



A- 4 




row 

column 

16 

5 

4 

4, 5 
4, 4 

44 

no 

9 

A 2 

5 $*■ 
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2-9 A factual student response is followed by a teacher 
question 13 times. The number 13 would be placed 

in row , column . Enter the tallies in 8,4 

the proper cell. 

2—10 A factual student response is followed by teacher 
directions 12 times. The number 12 is placed in 

the cell. Enter the tallies in the proper 8 6 

cell. ’ 



INCOMPLETE PRACTICE MATRIX 



Figure 1 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


Total 


1 
























2 




2 ' 


2 




12 


5 


1 










3 




1 




3 


1 


2 












4 




2 








6 


1 










5 




2 






18 


14 












6 


• 


2 


• 


7 


6 


16 




23 




6 




7 










1 














8 
















28 








9 
























10 








4 




5 








14 




Total 

i 
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PHASE III - INTERPRETING CLASSROOM INTERACTION FROM THE "FRINGE” 

MATRIX DATA 

We now have an idea of how raw data is entered into the 
matrix. We can now quantitatively analyze the data that we have 
in the matrix by working with the "fringe" or border areas of the 
matrix. (Refer to Figure 2.) 



The first data that can be taken from the fringe area is 
the percentage of the total interaction in each column. This is 
found by dividing the number of tallies in each column by the total 
number of tallies which in this case is 343. (See Figure 2.) 



3-1 The greatest proportion of interaction is found in 
column . 



8 



3-2 The percentage of interaction in column 8 is . 28% 

3-3 The smallest proportion of interaction is found in 

column and . 1> 9 



3-4 The percentage of interaction in column 1 is 
In column 9 the percentage is also . 



0 

0 



Referring to Figure 2, we notice that we have 
calculated percentages for each column. From the column 
percentages we can find the percentage of "teacher talk" 
by adding columns 1-7. 

3-5 The percentage of teacher talk is . This means 

that the teacher talked 66% of the time in that 
class. 



66 % 



3-6 We can find percentage of student talk by adding 
columns and ,. 

3-7 The percentage of student talk is _ 

3-8 The percentage of total talk in the classroom can 
be found by adding . 



8, 9 
28% 



rows 1-9 
or teacher 
talk plus 
student 
talk 
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In order to determine the nature of the teacher’s influ- 
ence on the students, we calculate the ratio of the indirect 
teacher influence (sharing, praising, accepting, questioning) to 
the direct teacher influence (lecturing, commanding, criticizing). 
This is called an I/D Ratio. The indirect teacher influence 
columns are 1 through 4, and the direct teacher influence columns 
are 5 through 7. 



I/D = 1+2+3+4 
5+6+7 



To calculate the total number of indirect tallies we add 
columns 1 through 4. 



3-9 The total number of indirect tallies in the sample 
matrix is . 

3-10 The total number of direct teacher influence 
tallies in the sample matrix is . 

3-11 If we divide the indirect tallies by the direct 

tallies, we get an ratio of . For 

example, if a teacher has 213 tallies in rows 
1-4 and 5-7, this would give an I/D ratio of 213 
which equals 1. 213 

An I/D ratio of 1.00 means that for every 

indirect teacher-influence statement, there was one 

direct teacher-influence statement. 

3-12 A teacher had 200 indirect statements and 100 
direct statements. His I/D ratio is . 

3-13 An I/D ratio of 2.00 would mean that for every 
two indirect teacher-influence statements, there 
was direct teacher-influence statement. 

3-14 An I/D ratio less than 1.00 would mean that the 

teacher is more than indirect in his 

influence on the students. 

3-15 Therefore, a more indirect teacher would have a 

I/D ratio than a more direct teacher. 

(a) higher (b) lower 



113 

112 

I/D, 

1.008 



2 



one 



direct 



(a) 

higher 



rr 
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4 



m 



A revised I/D ratio, sometimes written as small i/d 
ratio, can be used to determine teacher emphasis on motivation and 
control. In this ratio, the number of tallies in columns 1, 2 
and 3 is divided by the number of tallies in columns 6 and 7. 



i/d = 1+2+3 
6+7 



3-16 



The revised i/d differs from the l/D ratio in that we 
omit columns and . 



4, 5 



3-17 The revised i/d ratio is less concerned with actual 
presentation of subject matter and more concerned 
with and . 

3-18 The revised i/d ratio for our practice matrix is 



3-19 The above revised i/d ratio indicates that the 

teacher used more teacher influence than 

teacher influence in motivating and con 1 

trolling the students because the i/d ratio is 
less than 1. 



motivation, 

control 



45 , 1-3 
66 6-7 



=.681 



direct 

indirect 



3-20 If the same teacher would have had a revised i/d 
larger than one, then we would say that in matters 
of motivation and control the teacher's influence 
was usually . . 



indirect 



Our final I/D is called the I/D ratio for rows 8 and 9 
or l/D 8,9. This ratio is found by adding the tallies in rows 8 
and 9, columns 1 through 4 and dividing this number by the tallies 
in rows 8 and 9, columns 5 through 7. The type of teacher reaction, 
direct or indirect, to student talk is recorded by this ratio. 



3-21 In columns 1-4 the tallies in rows 8-9 represent . (b) 

(a) The direct portion of rows 8 and 9 

(b) The indirect portion of rows 8 and 9 

3-22 In columns 5-7 the tallies in rows 8-9 represent . (a) 

(a) The direct portion of rows 8 and 9 

(b) The indirect portion of rows 8 and 9 



3-23 I/D 8,9 for the practice matrix is 

* 

3-24 The I/D 8,9 gives us some idea of the type of 
teacher response to student » 




2.31 



A- 8 



talk 
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3-25 The high I/D 8, 9 for our practice matrix would 

indicate that the teacher used influence 

more than _____ influence in his responses to 
student talk. 



COMPLETE PRACTICE MATRIX "FRINGE" DATA 

Figure 2 

TEACHER I 







1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


Total 




1 




* 


















0 


INDIRECT 


2 




2 


2 


16 


12 


5 


1 








38 


INFLUENCE 


3 




1 




3 


1 


2 










7 




4 




2 




11 


4 


6 


1 


44 






68 


DIRECT 
INFLUENCE ■ 


5 




2 




9 


18 


14 








3 


46 


6 




2 




7 


6 


16 




23 




6 


60 


7 








5 


1 












6 


STUDENT 

TALK 


8 




29 


5 


13 


4 


12 


4 


28 






95 


9 






















0 


SILENCE 


10 








4 




5 








14 


23 




T 


0 


38 


7 


68 


46 


60 


6 


95 


0 


23 


343 




% 


0 


11 


2 


20 


13 


18 


2 


28 


0 


6 








INDIRECT 




DIRECT 




STUDENT 


S 








INFLUENCE 




INFLUENCE 


TALK 


I 

L 


























E 


























N 


























C 


























E 





Total Talk = 94% I/D = 1.008 

Teacher Talk = 66% i/d = .68 

Student Talk = 28% I/D 8,9 = 2.31 



indirect 

direct 



A- 9 
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PHASE IV - INTERPRETING INTERACTION PROM THE "CORE" OF THE MATRIX 

Now that we have examined the interpretations that can 
be made from the "fringe" areas of the matrix, we can turn to inter- 
pretations that can be made from the buildup of tabulations in 
individual cells and certain areas within the matrix. For this 
purpose we will use Figure 3 which outlines particular areas of the 
matrix with which we are concerned. We can then refer to the data 
in Figure 2 to give these areas practical significance. 



If you will look at Figure 3, you will see a shaded area 
in the form of a cross in rows 4 and 5, columns 4 and 5. This area 
is called the content cross. Tallies in this area represent teach- 
er lecture or teacher asking for information. Therefore, this area 
indicates teacher emphasis on subject matter. . 



4-1 



The "content cross" covers rows and horizon- 
tally and columns and vertically. 



4, 5 
4, 5 



4-2 We also notice that there is an area A at the inter- 
section of the cross. This area shows us the amount 
of extended teacher and . 

4—3 Turn to Figure 2. Looking at area A, we notice that 
the 5,5 cell contains 18 tallies. This means that 
there’ were 18 pairs of 5,5 combinations or 18 
instances of extended • 



lecture, 

questions 



teacher 

lecture 



4-4 An interaction such as this, lecture followed by 
lecture or praise followed by praise, is called 
"steady-state." Therefore, the 5,5 cell would be 
called a cell. 

4-5 Another "steady-state" cell in the "content cross 
is the cell. 

4-6 Are there any other "steady-state" cells within- 
the "content cross"? (a) Yes (b) No 



steady 

state 




No 



There are ten "steady-state" cells in the entire matrix. 
These are the 1,1, 2,2, etc., through 10, 10 cells from the upper 
left diagonally to the lower right of the matrix. Entries xn 
"steady-state" cells indicate that the speaker has remained in a 
particular category for more than 3 seconds. All other cells are 

"transition" cells. 
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4-7 


The 


5.4 cell is a 


cell. 


4-8 


The 


only other "transition" 


cell in area A is 




the 


cell. 





transi- 

tion 

4,5 



4-9 The 5,4 cell shows a transition from teacher 
lecture to teacher 



question 



4-10 The 4,10 cell shows a transition from 

to 



teacher 
question 
| silence 



Focusing our attention to the right-hand side of the 
matrix in Figure 3, we see area B. This is the area of student 
response. By looking at the tallies in the separate cells in 
columns 8 and 9, rows 1 through 7, we can tell what stimulated or 
initiated student talk. 



4-11 In order to determine what initiated student talk, 

we should look at columns and , rows 

through . 



8, 9 

1 , 7 



4-12 The 44 tallies in the 4,8 cell in our practice 

matrix (Figure 2) indicate that 44 times student 
talk was initiated by teacher . 



questions 



4-13 The 44 tallies in the 4,8 cell indicate memory- 

type teacher followed by factual, narrow 

student . 



questions 

answers 



4-14 The 23 tallies in the 6,8 cell indicate teacher 
followed by student . 



directions 

responses 



4-15 The 8,8 cell is a 



4-16 The 28 tallies in the 8,8 cell indicate amount 

of student responses that were longer than 

seconds in duration. 



"steady- 
state or 
extended 
student 
talk 

3 
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Just as area B in Figure 3 shows what initiated student 
talk, area C (rows 8 and 9, columns 1 through 7) shows the type 
of teacher response that followed student talk. 



4-17 Since all of the student responses in our practice 
matrix (Figure 2) were factual, narrow responses, 
all of the tallies in area C are in row . 



8 



4-18 In Figure 2 the teacher has usually responded to 
student answers with praise. We find this by 
looking at cell . 

4-19 Cell 8-, 2 has tallies. 

4-20 Cell 8,3 has __ tallies. 

4-21 If we compare cells 8,2 and 8,3 in Figure 2, 

we would say that the norm for this teacher is 
• 

(a) The teacher uses student ideas, elaborat- 
ing and expanding on them and integrating 
them into the lesson. 

(b) The teacher praises the students for giving 
the predetermined answer and then continues 
with his lesson. 



8,2 

29 

5 



b 



Moving next to the area designated as D in Figure 3, 
which is referred to as the extended direct area, we have cells 
showing criticism — directions sequences. This area also shows 
the vicious circle sequence in which the teacher gives a direction 
the student resist, the teacher criticizes and gives another direc 
tion, etc. 

4-22 Tallies in the ’’vicious circle” reflect upon . a 

(a) classroom r anagement and control 

(b) subject matter content 



4-23 The practice matrix (Figure 2) shows that 

(a) students did resist directions and 
were criticized 

(b) students did not resist directions 



b 
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MATRIX CODE AREAS 



Figure 3 




CROSS 



CONTENT 

CROSS 



STEADY-STATE 

CELLS 



4-24 The term "vicious circle" • 

(a) would apply to the practice matrix (Figure 2) 

(b) would not apply to the practice matrix 
(Figure 2) 



4-25 Area E in Figure 3 shows us the amount of extended 

teacher influence. 

(a) indirect (b) direct 



indirect 



4-26 The tallies in the 2,3 cell indicate a transition 

in teacher behavior from to _ 

of student ideas. 

4-27 Judging from the number of tallies in the 3,3 cell 
in Figure 2, we can assume that teacher acceptance 

and use of student ideas occurred • 

(a) frequently (b) infrequently (c) never 



praise 

accept 

ance 



c 
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COMPLETE PRACTICE MATRIX "FRINGE” DATA 



Figure A 



TEACHER II 







1 


2 


3 


A 


5 


6 


7 


8 


9 


10 


Total 




1 




1 






1 












2 - 


INDIRECT 

INFLUENCE 


2 


1 


A 


5 


6 


2 


1 






A 




23 


3 




1 


9 


3 


A 


2 






3 




22 


A 








A 


1 


1 




22 


A 


3 


35 




5 




1 


1 


9 


27 


A 






8 




50 


DIRECT 

INFLUENCE 


6 




1 




2 


2 


3 




2 


2 


1 


13 


7 










1 


2 


1 








A 


STUDENT 


8 




8 


5 


7 


6 






26 






52 


TALK 


9 


1 


7 


O 

JL 


1 


6 




3 




9 


2 


31 


SILENCE 


10 








3 








2 


1 


2 


8 




T 


O 


23 


22 


35 


50 


13 


A 


52 


31 


8 


2A0 




% 


1 


9 


9 


1A 


21 


6 


2 


22 


13 


3 






• 


INDIRECT 




DIRECT 




STUDENT 


S 

I 








INFLUENCE 




INFLUENCE 


TALK 


L 





E 

N 

C 

E 



Total Talk = 97% 
Teacher Talk = 62% 
Student Talk = 35% 



I/D = 1.20 
i/d = 2.8 
I/D 8-9 = 2.06 
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PHASE V - PROBLEMS INVOLVING THE COMPARISON OF TWO MATRICES 
(FINAL EXAMINATION) 

5-1 Which cell is used to determine extended teacher lecture? 

(a) 4,4 (b) 5,5 (c) 6,6 

In working the following problems, you will compare our 
practice matrix (Figure 2, Teacher 1) with the sample matrix 
(Figure 4, Teacher 2). 

5-2 Which teacher, comparing teacher No. 1 with teacher No. 2, 
has more extended lecture? 

(a) teacher 1 (b) teacher 2 

5-3 Which teacher is the more indirect? 

(a) teacher 1 (b) teacher 2 

5-4 Does an indirect teacher necessarily spend less time on 
extended lecture? 

(a) yes (b) no 

5-5 Which teacher asked more narrow, factual questions which 
were followed by factual student replies? 

(a) teacher 1 (b) teacher 2 

5-6 Which teacher used more extended commands or directions? 

(a) teacher 1 (b) teacher 2 

5-7 Which teacher has more lecture followed by student talk? 

(a) teacher 1 (b) teacher 2 

5-8 Which teacher is more inclined to encourage or praise a 
student response? 

(a) teacher 1 (b) teacher 2 

5-9 In which ratio do these two teachers differ more? 

(a) I/D (b) revised i/d 

5-10 This indicates a greater difference between the two in method 
of: 

(a) content presentation (b) motivation and control 

5-11 Immediately following student talk, which of the two teachers 
responds more indirectly? 

(a) teacher 1 (b) teacher 2 
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5-12 The answer to question 5-11 is determined by the: 

(a) I/D ratio (c) revised i/d ratio 

(b) I/D 8,9 ratio 

5-13 From the I/D 8,9 ratio we can say that: 

(a) both teachers respond more ind: ectly than 
directly to student- talk. 

(b) only teacher 1 responds more indirectly 
than directly to student talk. 

(c) only teacher 2 responds more indirectly than 
directly to student talk. 

(d) both teachers respond directly to student talk. 

5-14 The big difference between the two teachers in area C 

(teacher response to student comments) can be seen by com- 
paring: 

(a) the total number of tallies in the area for 
both teachers 

(b) row 9 of both teachers 

(c) the 8,8 cell of both teachers 

5-15 Looking at area D, we can see some evidence of the "vicious 
circle" in* 

(a) teacher 1 (b) teacher 2 (c) neither 

5-16 In Area E (extended indirect influence) we can see that 

teacher 2 has many more tallies than teacher 1, but the sig 
nificant difference in this area can be seen in the: 

(a) 2,2 cell (b) 2,1 cell (c) 3,3 cell 

5-17 Judging from the two matrices and from what you have learned, 
which teacher appears to be the more flexible in his teach- 
ing behavior? 

(a) teacher 1 (b) teacher 2 

5-18 In comparing the two matrices, it would seem that the students 
in which class had a greater freedom of speeehand were not 
afraid to present their ideas to the class? 

(a) teacher l*s class (b) teacher 2’s class 

5-19 Which teacher asked a question that was followed by silence? 

(a) teacher 1 (b) teacher 2 

5-20 Which teacher uses questions more often during his lectures? 

(a) teacher 1 (e) both use questions equally 

(b) teacher 2 during lecture 
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ANSWER SHEET 
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APPENDIX B 

HAYES PUPIL-TEACHER REACTION SCALE 



DIRECTIONS 



Please read these instructions first. Your sincere, 
thoughtful help will be appreciated. 



Do not place any marks on this paper. On the IBM card 
print your name (last name first), your teacher's name, school 
name and today's date. Then indicate your reaction to each item 
on the IBM card with the electrographic pencil which has been 
provided. In using the electrographic pencil, please darken 
heavily the entire space for each answer which you select. 



YOUR TEACHER AND SCHOOL ADMINISTRATORS WILL NOT SEE 
YOUR A NSWERS . The reaction of your entire class (as a group) 
will be given directly to your teacher by the Harrisburg Research 

Team. 



DO NOT TURN THE PAGE UNTIL TOLD TO DO SO.. 
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1. This teacher makes the lesson objectives clear in the first 
few minutes of the class: 







ci • 


Always 






b. 


Usually 






c. 


Sometimes 






d. 


Seldom or never 


2. 


She 


(he) 


really causes you to think: 






a. 


Most of the time 






b. 


Often 






c. 


Sometimes 






d. 


Seldom or never 


3. 


Her 


(his) 


explanations are: 






a. 


Extremely clear and to the point 






b. 


Very clear and to the point 






c. 


Adequate, might be better 






d. 


Often not clear or not to the poii 


4. 


Her 


(his) 


lesson materials are: 






a • 


Outstanding 






b. 


Very good 






c. 


About average 






d. 


Definitely below average 


5. 


Her 


(his) 


lessons provide very well for the 



and experience level of students: 

a. Always 

b. Usually 

c. About half the time 

d. Sometimes or seldom 



6. Her (his) instruction if very realistic: 



a . Always 

b. Often 

c. Sometimes 

d. Seldom or never 



APPENDIX B (CONTINUED) 



7. Her (his) instruction is: 



a. Extremely challenging 

b. Very challenging 

c. Somewhat challenging 

d. Not very challenging or usually un challenging 



8 . 



She (he) concludes lessons by: 




Capably emphasizing the main points 
Repeating the main points 

Abruptly stopping, but this does not bother 
Abruptly stopping and I often wonder what I 
should have learned during the period 



9. This teacher uses excellent examples to make ideas clear 



a. Most of the time 

b. Usually 

c. About half the time 

d. Sometimes or seldom 



10. This instructor is the very best I ve ever had. 



a. Agree 

b. Disagree 

11. This instructor is one of the best. ^ 

a. Agree 

b. Disagree 

12. Her (his) teaching is effective. 

a. Agree 

b. Disagree 

13. Her (his) lessons are at least average or better 

a. Agree 

b. Disagree 

14. Her (his) teaching is not quite adequate. 

a. Agree 

b. Disagree 
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15. Her (his) lessons are poor. 



a. Agree 

b. Disagree 

16. This teacher is very ineffective. 

a. Agree 

b. Disagree 

17. This teacher is the worst I’ve ever had. 



a. Agree. 

b. Disagree 

18. This subject is one of the best. 

a. Agree 

b. Disagree 

19. This subjeict is "okay”. 

a. Agree 

b. Disagree 

t 

20. This subject is dull. 

a. Agree 

b. Disagree 



APPENDIX C 



Commonwealth of Pennsylvania 
Department of Public Instruction 
Box 911, Harrisburg, Pennsylvania 17126 

April 15, 1967 



TO: Teachers Who Participated in the 1966-67 Feedback Study 



The attached Feedback Study Opinionnaire has been 
designed to obtain vital information on the opinions of each 
teacher who participated in the study. 



Directions : 

Please write your name and the treatment number to which 
you were randomly assigned in the space provided at the top of the 
opinionnaire. Read each statement carefully. If you agree with 
the statement, place a check-mark (*''') opposite the statement in 
the space beneath "Agree." If you disagree, place a check-mark 
opposite the statement in the space beneath "Disagree." If the 
statement pertains to some aspect of the study that did not apply 
to you, place a check-mark opposite the statement in the space 
beneath "Not applicable." 

Please react to the statements sincerely and frankly. 

The identification of teachers with their opinions will be kept 
confidential. 
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Name 



Treatment ' 



School 



Section A: Beneficial Aspects of the Study 

1. The feedback I received provided me with a 
critical analysis of my teaching. 

2. The feedback pointed out my teaching strengths 
and weaknesses . 

3. The feedback provided me with an objective 
basis for improving my teaching. 

4. I gained new insights into my teaching from 
the training I received in interaction anal- 
ysis. 

5. Student interest in the observed subject was 
increased. 

6. Test score feedback increased my understand- 
ing of my pupils. 

7. Participation in study provided very little 
or nothing that I consider useful or helpful 
to me. 



Section B: Disliked Features 

8. I did not really want to participate in this 
study; I was made to feel that I was obligated 
to participate. 

9. I gained little useful information from the 
feedback I received to help improve my 
teaching. 

10. I felt ill-at-ease when my class was observed. 

11. During the observations by the co-investigator, 
jay pupils were less responsive to my teaching. 
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Section C: Method of Feedback 

12. I would have preferred the receipt of written 
feedback rather than face-to-face feedback. 

13. I would have preferred the receipt of face- 
to-face feedback rather than written feedback. 

14 o Face-to-face conferences helped me understand 
the feedback reports I received. 



Section D: Attempted Changes in Teaching 

Performance 

Based on the feedback I received, I attempted to: 

1 j * Continue my normal teaching patterns since 
the feedback I received suggested no changes 
in my teaching that I felt would be profit- 
able in terras of improving pupil achievement 
and/or pupil attitudes. 

16. Make the objectives of the lessons clearer to 
my pupils. 

17. Make my explanations and instructions clearer. 

18. Improve my lesson materials. 

19. Provide for the needs, interests and experi- 
ence level of my pupils. 

20. Provide more realistic and challenging 
instruction. 

21. Better conclude my lessons. 

22. Use a greater variety of teaching methods. 




* 

J 



C-3 



3 



APPENDIX C (CONTINUED) 



i 




23. Accept the feelings of my pupils more than I 
previously did. 

24. Provide more praise and encouragement of my 
pupils for their responses and ideas. 

25. Accept and use the ideas of my students. 

26. Ask a lesser amount of memory-recall questions 
and a corresponding larger amount of thought- 
provoking questions. 

27. Lecture less and encourage more active student 
verbal participation in the topics under dis- 
cussion. 



Section E: Interaction Analysis 

28. During the orientation/ training meetings in 
September, I received an adequate understand- 
ing of the rationale and the purpose of this 
study. 

29. I received adequate training in interaction 
analysis to understand the feedback reports 
I received. 

30. Too much was covered in too short a time at 
the orientation/ training meetings. 

31. Instruction was not sufficient for accurate 
interpretation of the matrices. 

32. There was too much lapse between the train- 
ing sessions and receipt of feedback. 

33. Since the training I received was inadequate 
for me to understand the feedback reports, I 
had no meaningful basis for attempting to 
change my teaching. 
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34. An adequate sample of my total verbal teach- 
ing behavior was observed in the fall to give 
me a basis for making decisions on whether to 
change my verbal teaching patterns. 

35. The lesson content of the subject selected 
for observation restricts the amount of 
teacher- student verbal interaction. 

36. In my teaching situation, I feel that direct 
teaching is generally more effective than 
indirect teaching. 



Section F: Student Ratings 

37'. Most of my pupils accurately rated my teaching 
performance. 

38. The pupil ratings of my teaching provide a 
reliable index of my competence as a teacher. 

39. Sixth-grade pupils are too immature to dis- 
tinguish good teaching from poor teaching. 

40. Most of the pupil ratings of my teaching were 
probably influenced by such factors as person- 
ality differences, marks, discipline, their 
attitudes and general scholastic ability. 



Section G: Student Test Scores 

41. The test score information increased my under- 
standing of my pupils. 

42. Knowledge of my pupil test scores was influ- 
ential in changing my lesson plans and teach- 
ing. 
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Agree 


Disagree 


Not 

Applicable 


(Please Check) 









Section H: Recommendations 

43. The benefits of pupil ratings of teaching 
performance in the improvement of teaching 
are sufficient to suggest their widespread 
use by upper-intermediate grade teachers. 

44. The benefits of teacher— pupil verbal inter- 
action analysis suggests training in its use 
in inservice programs designed to improve 
teaching behavior. 
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TEACHER REACTIONS 



BENEFICIAL ASPECTS OF THE STUDY 

As can be seen in Table 1, a very large majority of the 
teachers felt that the feedbapk they received provided them with 
critical analyses of their teaching, pointed out their teaching 
strengths and weaknesses and provided them with objective bases 
for improving their teaching. Only five teachers felt that they 
gained little or nothing from their participation in the study. 

Almost all of the teachers reported that they gained new 
insights into their teaching as a result of the training they 
received in the Flanders system of teacher-student verbal inter- 
action analysis. Less than one-half of the teachers felt that 
student interest in the academic subject selected for observation 

was increased. 



DISLIKED FEATURES OF THE STUDY 

The sample was selected randomly from lists of volunteer- 
ing sixth-grade teachers which were submitted by the seven partic- 
ipating school districts. Even so, according to the teachers 
reactions to statement 8 of Table 2, 15 percent of the teachers 
did not want to participate but did so because they were made to 
feel they were obligated to take part. 

Although most of the teachers agreed with the statement 
that the feedback provided them with useful information to help 
improve them as teachers, there were 13 teachers who disagreed. 

It should be noted, however, that over one-half of the teachers 
who disagreed received only the student achievement test scores 

a9 feedback. 

Only one of the 12 teachers who felt ill-at-ease when 
his classes were observed felt that his pupils were less responsive 
to his teaching during the observations. Presumably, the respon- 
siveness of the pupils did not seem to contribute to whether or 
not the teachers felt ill-at-ease, nor did the discomfort some 
teachers felt when they were observed seem to affect the normal 
responsiveness of their pupils during the observations. 



D-l 



F l 
i 




I 



j 




D~2 







/-s 



X 

H 

Q 



ft 

ft 



04 



O J 00 

H vo 






04 CO 



04 00 



in LO 



CO r-s 

H VO 



Os CO 



04 00 
tH 



H CTv 
H 



CO Os 



04 00 
rH vO 



m in 

rH 



CO Os 



04 CO 



CM 00 



Qv 

VO 



O* 00 



m m 

H 



04 oo 



04 00 

rH 





0 


Q) 




0 




0 


QJ 




0 




U 


M 




M 


QJ 


00 


QJ 60 


0 


W) 


QJ 


0 


a (Q 


0 


0 


M 


CO 


M (/) 


M 


0 


60 


•H 


60 *H 


&0 


•H 


<a 


0 


<3 © 


< 


o 



CO 

^ cd 



u •« 



<+H 

(3 W 

♦H 

H U 
3 0 
m *ja 
0 no 
0 
0 
44 
0 

H 0 
44 

44 44 



0 

I"* 

r 

s 

Q* 



3 Sfl 



A 

Q) 



J 

QJ 

,a 

& 

*» 

*» 

0) • 
w r d 
cfl a) 
aj > 
I M 
44 GJ 
<tj m 





• 






H 


n 


0 


H 


Q 


44 


• 


CO 






0 


44 u 


H 




0 


• 


cd 






u 


• 


•H 


(0 


a 


0 


& 




0 


44 


*0 60 




0 




44 






3 




a» a 


44 


{* 


*x) 


0 


H 


•M 


*H 


fl 


> *H 


H 




*H 


pu 




• 


0 


a 


•H 43 


0 


co 


*0 


♦H 


44 


QJ 


M 


*H 


QJ O 


44 


CO 




a 


nJ 


44 




44 


a co 




0 


H 


♦H 


43 


flj 


H 


0 


QJ QJ 


H 


i*4 


<*■ 

•« 


44 


44 




m 


0 


M 44 


m 

m 


a 



00 



o\ 



© 

H 

D-3 



H 

H 




APPENDIX D (CONTINUED) 



METHOD OF FEEDBACK 

The reactions of the teachers to the two methods of 
feedback employed in the study (face-to-face and written) are 
shown in Table 3. Of the 40 teachers who received their feedback 
reports in face-to-face conferences with the co-investigators, 
only six of them would have preferred their feedback reports by 
mail. While six teachers would have preferred their feedback to 
be mailed to them, only two of the 40 teachers felt that face-to- 
face feedback conferences did not help them understand the feed- 
back they received. On the other hand, 29 of the 40 teachers who 
received their feedback reports by mail would have preferred their 
feedback through face-to-face conferences. 



CHANGES ATTEMPTED IN TEACHING BASED UPON FEEDBACK 

Statements 15 through 27, which appear in Table 4 
together with the reactions of the teachers to them, were included 
in the opinionnaire to determine if and what changes in teaching 
were attempted by teachers that were based upon the feedback they 
received. Statements 16-22 pertain to changes attempted which 
were suggested by feedback on the Hayes Pupil-Teacher Reaction 
Scale. Statements 23-27 pertain to changes attempted that were 
based upon interaction analysis feedback. 

Fifty-one teachers reported that they attempted at least 
one change in their teaching in contract to only nine teachers who 
reported that they attempted no changes. On the basis of the 
responses of the teachers to each statement, it is apparent that 
most of the teachers who received feedback on their teaching per- 
formance felt that there were areas in which their teaching could 
be improved and made efforts to change their teaching. 



INTERACTION ANALYSIS 

The reactions of teachers to statements which pertained 
to various aspects of interaction analysis are shown in Table 5. 

Two fall orientation/ training meetings were held to 
train each teacher in the use of the Flanders system of interaction 
analysis and to provide each teacher with an understanding of the 
rationale and mechanics of the study. Seventy-eight percent of the 40 
teachers who received interaction analysis feedback felt that they 
had received adequate training to understand the feedback they 
received. 



ft 





V 


& CO 
H 

<0 

p 




















v£> 


<t 0 




. CT\ « — 1 O 




00 


CM O 






0 




co <t 




CM H •<f 




CO 








H 




















<t 




















P 




















d 




















0) 




















6 


<N 


00 0 




h> fO 0 




0 


! O 






4J 

cd 

0) 

P 

H 

CO 

P 

d 

§ 




rH 




1 — 1 




rH 


rH 






rH 


a> 0 




00 CM 0 




ON 


rH O 






4J 




rH 




H 






rH 






cd 




















0) 




















P 




















H 




















CM 




















4J 




















d 


















1 


<D 




















6 


CM 


00 0 




NCOO 




O 


1 O 






4J 




rH 




rH 




rH 


rH 






cd 




















CD 




















P 




















H 
















o 




















w 


X! 


















D 


O 


1 — 1 


















<J 


















H 


PQ 


4J 
















H 


O ' 


d 
















J3 


co W 


a) 
















O 


w 


B 


rH 


<T> O 




N CO O 




CT\ 


rH O 


O 


W fe 


4J 




H 




H 






rH 


v 


t-3 


cd 


















pq pm 


a) 
















Q 


< 0 


p 
















XJ 


H ' 


H 
















0 


















H 


0 


















O 


sc 


0) 




a) • 




QJ • 






a) • 


§ 


H 


CO 




a) *q. 




qj a. 






a) p< 


w 


pr*! 


d 




p a. 




>-l p4 






P P4 


P-4 

Py 


§ 


0 


0) 


bO <J 




(U 00 <J 




0) 


b0 <d 




Ph 


a) 


cd 




(0 cd 




a) 


cd 


53 




CO 


P 


CO 4J 




M W 4-1 




p 


CO P 






Q) 


bJO 


•rH O 




00 -H O 




bp 


*H O 






P3 


«S 


Q |25 




<1 O 13 




< 


P S 




















1 

*d 














0 






<D 








CD 






(0 0) 






Q) 








rd 


a 




rO 4-1 














P 


cd 




4-1 4-1 




CO 


•» 










,ri d 




•rH 




<D 


0 • 








•d 


nd O 




*d 0) M 




0 


rC T3 








0) 


a) cd 




aj 0 £ 




d 


4-1 a) 








p 


(U 4H 




M cd 




a) 


X 






p 


P 


4H I 




u m 0 




p 


nd *h 






d 


0) 


O 




aj 1 cd 




0) 


p ^ 






Qj 


4H 


d 4J 




M4 042 




<4H 


.Cd O 






B 


0) 


0) 1 




Q) 4J 4J 




d 


4-1 aj 






a) 


P 


p cd 




U 1 




0 


CO JH 






p 


P. 


p 0 




cu <u u 




0 


M 






cd 


•ri cd 




0 <u 






QJ H 






p 


a) 


P MH 




05 cd ,£3 




Q) 


Td 






c/3 


!> 


& 




> m 4J 




O 


0 CO 








cd 


d 




cd cd 




cd 


P 4-1 










4H Cd 


tm 


HP «4H V4 


•M 


m 


N 








O rd 


• 


0 


• 


1 


QJ O 










p 


44 


’d 44 44 


0 


S 04 








■3 


p 


a 


rH P O 


Cd 


p 


QJ 








Ph u 


cd 


P P 4 cd 


cd 


1 


Td J-4 








O 


*H OJ 


rO 


O *H HP 
*2 <15 Td 


rQ 


Q) 


QJ 








|2 


a) 42 


*d 


*d 


O 


04^4 










a p 


a) 


a QJ 


CD 


cd 


H O 








M 


a) cd 


a) 


1 — l CL> CL) 


Q) 


(H 


QJ P 




„ 




•» 


P U 


m 


s j-i m 


HH 




H3 H3 



• • • 

CM CO 



D-5 



o 

ERIC 



CA 



cd 

4J 

o 

IH 



U 

C 

<U 

S 

u 

cd 

<u 

u 

H 



ro 

U 

C 

§ 

cd 

<u 

u 

H 



CM 

4 J 

P 

CD 

S 

4 J 

cd 

CD 

u 

H 



4 -J 

p 

0 

4 J 

cd 

CD 

H 



CL) 

CA 

P 

O 

P. 

CA 

CD 

Pi 



U 

s 

§ 

44 

d 

+j 

CO 



Oi H O 

in <n 



i o 

(N 



ro r-. i 



CM oo | 



MJ- vD | 



a) • 
a) a 
JH (X 
a) oo 
a) d 

lH CO 44 

00 *H O 
<1 O 13 



CD 

O 




CL) H 


00 


p 






d 


■H 




& 


•rJ 


CA 


o 


nd P 


> 




d 


H Pj 


O 


CA 




3 


JH 




X) 44 

( 1 ) 0(0 
•M (0 

B **§ w 

a) a) co 

4J o) ai 

+j m oo 

10 • s 

H Jd 

C 44 o 



m 

H 



o o o 

(O H <f 



I O 

CM 



<t VO | 



I O 

CM 



VO si* | 



a) 

a) 

M 



Op *H 
<1 Q 



a) 

QJ 

M _ 
00 < 
(0 
CO 



Pu 



U 

£ 



QJ pj 
rP O 
w to: 

CO • 

a) a) co 

^ H H 
(0 *H 

0 a) a 
•d d 
o +j a 

44 

m >, 

*8 ° 6 
•U CO O 
Pi QJ 44 

S > 

0) *H JH 

4-1 4-1 QJ 

■WOM 
(0 Q) (0 
*n d) 
W fd rH 

s o o 



VO 



o o o 

(O H M 



I I o 

CM 



m m i 



I o 

CM 



m m i 



a) • 

a ) ex 

. u Q 
0 ) 00 <1 

0) (0 
14 (0 44 
Op »H O 

< Q Jz; 



CO 

d 

o 

•H 

44 

O 

0 JH 

4 J 
0) CO 

^ d 

S * H 

X ) 

o d 

44 (0 

TO CO 

0 ) d 

44 O 

Is 5 - 

QJ d JH 
44 d 0 ) 
•M (0 M 
d rH d 

h B*5 

S QJ O 



n- 

H 



00 CM O 
N H <f 



I I O 

c . 



co i 



I o 

CM 



m m i 



qj . 
qj ex 

M 04 

0) 00 <4 

QJ d 
»H CO 44 

00 *H O 

<1 O 13 



& 



QJ 

> 

O 

1-4 

& 

n = 

•H • 
CO 

O 'H 
44 d 
_ *H 
Tl J-4 
QJ QJ 
44 44 

8*1 

QJ 

44 d 
44 O 
d CO 
CO 

H QJ 



oo 



Q> rl O 
OHM’ 



I I o 

CM 



CM 00 I 



O 

CM 



r> co | 



QJ . 
QJ 04 

J -4 04 

QJ 00 <1 
QJ d 
J -4 CO 
00 «H _ 
<1 Q 13 



44 

O 



CO 
H 
•H 

§* 

04 

I’ 

CA 

O U M-| 
M CA O 
p 4 CD 
U H 
o > a ) 

4J > 

P 0) 
Td *H H 
0 ) 

4J 



M 

O 

m 

<D 

*rd 

£ 



o 

44 



t 



CA 

nd 



0 ) 

o 

P 



0) QJ <D 
4J <D 
4J ti 

cd 

<D 

H & 

: 4J Q) 



CT> 



H 

M 

<d 



o o o 

(OHM 



I o 

CM 



<t m i 



I o 

CM 



m m i 



qj • 

QJ 04 
J-4 04 
QJ 00 <1 
QJ d 

M CO 44 
00 *H O 
<T Q 53 



QJ 

J -4 

Q 

0 00 

d 



QJ 

X ) 



H 
00 
•H d 
p> QJ 
O rH 
J -4 H 
04 d 
id 

o o 

44 = 

XJ • 
x) d co 
qj d d 

44 o 

a o *h 

0 *H 44 
QJ 44 O 
44 CO d 
44 .H JH 
d H 44 

d co 
H QJ d 
S J -4 -Pi 



o 

CM 



D-6 



o 

ERLC 



w 



Q 

X! 

M 



W 

P 

Si 



XJ 
a 

<3 
eq 
P 
W 

e 

§ 

6 

S 

P C/3 



|P 

H O 

H S 

§ a 

8g 

st H 

W 525 
P H 



§ 



. P 
H W 



4-1 

O 

H 



4J 

d 

8 

4J 

cD 

0) 

U 

H 



CO 

4J 

d 

8 

4J 

cD 

0) 

V4 

H 



CM 

4J 

d 

8 

4J 

cD 

cu 

M 

H 



4J 

d 

8 

4J 

CD 

cu 

M 

H 



CU 

co 

d 

o 

P 

co 

cu 

P4 



4J 

d 

8 

a) 

4J 

cD 

4J 

CO 



Ch rH O 
CM t — I 



I I o 

CM 



m m i 



I i 



o 

CM 



vo i 



cu . 
cu p 

M P 
CU 00 <1 
CD cD 
Vi CO 4J 
OO *H O 

<] p s 



cu 

x* 

3 

rH 

a 

d 

o 

a 

H 

cu 

4J 

4J 

cu 

fO 

o 

4J 



XI 


•9 


<U 


« 


4J 

t 


co 

d 

o 


(U 


CO 


4J 


CO 


4J 


(U 


CD 


rH 


H 





VO O 
CM H M- 



t O 
CM 



co r^s | 



i I 



o 

CM 



co r> 

H 



cu • 
cu p 

Lj cl 

CU 00 <J 
CU CD 
H CO 4J 
00 vH O 
< P 525 



U Z 
0 ) • 
4J CO 
CD Xl 
<U O 

N rd 

00 4J 
CU 

CD 0 



<U 00 

•g 

■8 
cD 
CU 
4J 



S 

o 

4J 



XJ 
(U 
4J MH 
O 



& 



0) >> 
4J 4J 



4J 

CD 



CU 
•H 
U 
H cD 
- > 



CM 



CM 

CM 



SO O 
CM H M* 



I I O 
CM 



CM CO 
H 



<t VO 

H 



I I O 

CM 



<U • 

cu p 

M P 

cu oo <3 

CU cD 
M CO 4J 
50 *H O 

<i P js; 



o 

4J 



nd 
CU 
4J 

& 

a) m 

4-1 o 



H • 

a, xi 
p *H 
p x 



S' 



4J 

CD 

H 



CO 

CM 



o 

CO J> 
00 CU 

d m 

H P 



r- co i 

CM rH 



I I I 



co i 



<± vo | 



i I l 



cu . 
cu p 

M P 

<u oo <3 

CU cD 
M CO 4J 
00 *H O 
<3 P is; 



H 








<U 








CU 


H 




0 § 


m 




<U 


cu 


d 


M 


m 


cD 


Q 


O 03 


rd 


rd 


0 


a> 


4J 


4J 


4-i 03 






<U 


P P 


4J 


<u 


XJ 


a) o 


P 


M 


•H 


0 p 


(U 


O 


> 


a) 03 


a 


s 


o 


60 0) 


a 


U 


P M 


cD 


CO 


P 


M 



o 

4J 

nd 

a) 

4J 

& 

(U 

4J 

4J 

CD 

H 



CM 



d 

O 

a 

g 

nd 



cu 

co 

>H 

cD 

M 

P 



M 

•H 

<U 

rd 

4J 

M 

O 

MH 

CO 

rH CO 
•H CD 
P CU 

d x 

P *H 



i — i o 

CO <t 



I I o 

CSJ 



St XI I 



I"* CO | 



I I o 

<N 



a) • 
a) p 
H p 

a) 60 <3 

p p 
H 03 4-1 
60 *H O 

<1 P 3 




m 

CN 



rH 03 O 
CO st 



I I o 

CM 



VO | 



lo m i 



i o 

CM 



CU • 

cu p 

H p 

cu oo <3 
cu cD 

M CO 4J 
00 *H O 

<3 P & 



CO 



4J 4J 

o 

H 0 



VO 

CM 



O 

a 

cD 

r d 

d 

cD 



d 

o 

•H 

4J 

CO 

CU 

d 

cr 



rl Q> O 
CO N^ 



I o 

CM 



CM CO I 



Ch rH l 



I I o 

CM 



CU . 

cu p 

H Pm 

<u oo <3 

(U CD 
H CO 4-1 
Op *H O 

<3 P tz* 









d 


















O 












cu 






•H 






1 






CO 






4J 






d 




9* 


d 




u 


CO 


M 


CO 


4-1 


d 


• 






(U 


(U 


a) oo 


CO 


CO 


•H 


d 


X* 




CO 


d 


60 pi 


(U 






O 


d 


• 


CO 


CT* 




rH 


cu 


d 


*H 


cD 


CO 


(U 




P AS 




p> 


o 


CO 




4J 


rH 


rH 


H O 


(U 


•H 


•H 


CO 


4J 


d 




rH 


r* 


H 


4-1 


4-1 


d 


p 


(U 


CD 


CD 


!>> O 


d 


a 


cD 


a 


CU 


Td 




O 


rH M 


4J 


cD 


p 


co 


a 


d 


44 


(U 


60 P 


a 






•H 


a 


4J 


CO 


M 


P 1 


cu 


(U 


a 


nd 


cD 


<0 


cD 


1 


•H 4-1 


H 


M 


•H 










h 


XI X3 




o 


4-1 


M 


O 




o 


H 


P 60 


o 


0 


H 


CU 


4J 


0 


4J 


o 


O P 


4J 




cD 


XJ 








0 


P O 




cu 


P 


d 


nd 


m 


X> 


0 ) 


03 rC 


XJ 


00 




d 


<U 


o 


(U 


0 


P 4-1 r 


(U 


CD 


H 




44 




4J 




M • 


4J 


u 


CD 


CO 




CO 


P MH 


M <4H 03 


& 


d 


rO 


a 



. . o 

a) a 



d 

cu 



4J 
4J 

cD 

r a 

= H 9 



CM 



M 

CU 

f> 

4J 

g 

nd 



•rH 

P 

O 

4J 

<U 

4J 



D-7 




APPENDIX D (CONTINUED) 



Well over one-third of all teachers agreed that too much 
was covered in too short a time at the orientation/ training meet- 
ings and that instruction in interaction analysis was not suffi- 
cient for thew to accurately interpret the matrices. In spite of 
this, only fo^J teachers in Treatment 3 indicated that they felt 
the training was inadequate for them to understand the feedback 
and they had no meaningful basis for attempting to change their 
teaching. 



Eight teachers reported that there was too much time 
lapse between the training sessions in interaction analysis and 
the receipt of feedback. Seven of these teachers were in 
Treatment 3. These teachers received their first feedback approx- 
imately six weeks after the last training session, whereas teach- 
ers in Treatment 2 received their reports within four weeks of the 
last training meeting. 

Teachers in Treatment 2 were observed four times in the 
fall. Only two observations were made of teachers in Treatment 3. 
More teachers in Treatment 2 than in Treatment 3 felt that an 
adequate sample of their total verbal teaching behavior was observed 
to form a basis for making decisions on whether to change thei^r 
verbal teaching behavior. Although 11 teachers reported thatjlan 
adequate sample of their verbal teaching behavior was not observed, 
almost all teachers did indicate that they attempted to change at 
least one aspect of their verbal behavior. (See Table 5 for 
teachers’ reactions to statements 15 and 23—27.) 

Forty percent of the teachers felt that the lesson con- 
tent of the academic subjects selected for observation restricts 
the amount of verbal interaction. The subjects and number of 
teachers who felt that the lesson content of these subjects 
restricts verbal interaction were: mathematics-14, English-8, 

science-5, reading-4 and social studies-1. 

Of the 17 teachers who agreed that direct teaching is 
generally more effective than indirect teaching in their teaching 
situations, 13 were males. (The ratio of male to female teachers 
in the study was 38 to 42.) In the academic subjects selected for 
observation, the average grade placement on the achievement pre- 
tests of pupils of nine of these teachers was less than 6.0. 



STUDENT RATINGS OF TEACHING 

The reactions of teachers to statements concerning 
student ratings of teaching performance are shown in Table 6. 
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APPENDIX D (CONTINUED) 



Although there were only four teachers who reported that 
most of their pupils did not accurately rate their teaching per- 
formance, there were 10 teachers who felt that the ratings do not 
provide a reliable index of their competence as teachers. Only 10 
percent of the teachers felt that sixth-grade pupils are too im- 
mature to distinguish good teaching from bad teaching. 

Over one-half of the teachers agreed that most of the 
pupil ratings were probably influenced by factors such as person- 
ality differences, marks, etc. In light of this it is somewhat 
surprising that 36 of the 40 teachers who received feedback on 
their pupil ratings of their teaching felt that most of their 
pupils accurately r.ated their teaching. 



PUPIL ACHIEVEMENT TEST SCORES 

Statement 41 in Table 7, concerning whether test score 
information increased the teachers' understanding of their pupils, 
and statement 6 in Table 1 are very similar. Even so, there are 
differences in the reactions of the teachers to each statement in 
all treatments except Treatment 3. Since all teachers received 
pupil-achievement test score feedback, it cannot be determined 
why nine teachers responded "Not Applicable" to statement 41. 
Slightly over one-half of the teachers agreed that knowledge of 
their pupil test scores was Influential in changing their lesson 
plans and teaching 



RECOMMENDATIONS FOR EDUCATION 

The responses of teachers to statements concerning 
recommendations for education appear in Table 8. 

Thirty-three of the 40 teachers or 83 percent who 
received feedback on pupil-teacher ratings reported that the 
benefits of the ratings in the improvement of teaching are suffi- 
cient to suggest their widespread use by upper-intermediate grade 
teachers. Slightly over 86 percent of all teachers in the study 
agreed that the benefits of teacher-pupil verbal interaction 
analysis suggest training in its use in inservice programs. 
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APPENDIX E (CONTINUED) 



TABLE 11 

ANALYSIS OF VARIANCE OF 
PRE-STANFORD CLASS ACHIEVEMENT MEANS 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


47514.15 




— 


Treatments 


3 


60.52 


20.17 


.03 


Type Feedback 


1 


.11 


.11 


.000 


Interaction 


3 


67.98 


22.66 


.034 


Within (error) 


72 


47385.54 


658.13 





TABLE 12 

ANALYSIS OF VARIANCE OF 
POST-STANFORD CLASS ACHIEVEMENT MEANS 


Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


52210.01 


mm 


— 


Treatments 


3 


67.46 


22.49 


.031 


Type Feedback 


1 


27.50 


27.50 


.038 


Interaction 


3 


49.17 


16.39 


.023 


Within (error) 


72 


52065.88 


723.14 





APPENDIX E (.CONTINUED) 



TABLE 13 

MEANS FOR REMMERS SCALE 
PRE-TEST 

! ■■ " 1 1 f a ■ 1 F I — — ■ 



Feedback 


1 


TREATMENTS 
2 3 


4 


Means for 
Total 




7.9 


7.5 


6.2 


7.1 






7.5 


7.7 


7.2 


7.6 






7.8 


7.7 


8.1 


7.5 






7.4 


7.6 


7.3 


7.9 






7.4 


8.2 


7.9 


7.6 




Face-to-Face 


. 7.8 


7.5 


7.7 


7.4 






7.3 


7.2 


7.8 


6.6 




* 


7.1 


.7.8 


7.2 


7.8 






8.1 


8.0 


8.2 


8.3 






7.9 


7.4 


7.8 


8.1 




Means 


7,6 


7.7 


7.6 


7.6 


7.6 




7.7 


7.7 


6.8 


8.0 






7.9 


7,9 


7.6 


6.6 






8.0 


7.8 


8.3 


7.8 






7.9 


7.8 


7.5 


7.4 






7.2 


7.8 


8.2 


7.9 




Written 


7.6 


7.8 


7.9 


7.7 






8.1 


8.0 


7.5 


7.7 






7.8 


8.0 


7.8 


7.5 






7.3 


7.5 


8.1 


7.3 




i ? 

1 

l 


8.4 


7.2 


8.1 


7.6 




\ 

Mejans 


7.8 


7.8 


7.8 


7.6 


7.7 


i 

Means for 












Treatments 


7.7 


7.7 


7.7 


7.6 


7.7 












(Grand Means) 
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APPENDIX E (CONTINUED) 
TABLE 14 





MEANS 


FOR REMMERS 
POST-TEST 


SCALE 










TREATMENTS 
















Means for 


Feedback 


1 


2 


3 


4 


Total 




8.0 


7.4 


7.5 


6.9 






7.6 


7.6 


7.1 


7.7 






8.2 


7.5 


8.2 


7.7 






8.0 


7.8 


8.2 


6.9 




Face-to-Face 


7.7 


8.3 


7.9 


7.9 






.7.3 


7.4 


7.9 


7.9 






6.9 


7.1 


8.1 


7.3 






7.7 


8.2 


7.4 


8.0 






7.9 


8.4 


8.1 


8.4 






7.6 


8.0 


7.5 


8.3 




Means 


7.7 


7.8 


7.8 


7.7 


7.7 




7,3 


7.5 


7.5 


7.8 






8.4 


7.3 


7.0 


7.1 






8.2 


7.8 


7.9 


8.3 






7.5 


8.3 


7.6 


7.5 




Written 


8.1 


7.5 


8.1 


7.9 






8.0 


7.9 


7.5 


7.5 






7.6 


7.8 


7.9 


7.9 






7.5 


7.8 


7.5 


8.1 






7.3 


7.9 


7.9 


7.1 






8.2 


7.5 


8.0 


8.1 




Means 


7.8 


7.7 


7.7 


7.7 


7.7 


Means for 












Treatments 


7.8 


7.8 


7.7 


7.7 


7.7 












(Grand Means) 




ERIC 



APPENDIX E (CONTINUED) 



TABLE 15 . 

STUDENT ATTITUDE TOWARD SCHOOL SUBJECTS (PRE-TEST) 
ANALYSIS OF VARIANCE 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


12.37 






Treatments 


3 


.24 


.08 


.50 


Feedback Level 


1 


.25 


.25 


1.56 


Interaction 


3 


.21 


.07 


.44 


Within (error) 


72 


11.67 


.16 


**■ 



TABLE 16 

STUDENT ATTITUDE TOWARD SCHOOL SUBJECTS (POST-TEST) 

ANALYSIS OF VARIANCE 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


11.81 






Treatments 


3 


0.02 


0.01 


0.06 


Feedback Level 


1 


0.00 


0.00 


0.00 


Interaction 


3 


0.14 


0.05 


0.31 


Within (error) 


72 


11.65 


0.16 


- 
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APPENDIX E (CONTINUED) 



TABLE 17 

MEANS FOR STUDENT-TEACHER RATINGS 
FIRST CYCLE 






TREATMENTS 
















Means for 


Feedback 


1 


2 


3 


4 


Total 




19.26 


18.92 


19.34 


17.76 






18.14 


19.74 


18.75 


17.59 






19.00 


18.22 


17.22 


19.57 






20.73 


16.69 


19.50 


20.00 




Face-to-Face 


15.68 


19.88 


20.41 


21.65 






17.78 


18.52 


19.80 


20.34 






15.43 


19.00 


20.24 


16.19 






16.50 


21.18 


17.31 


19.77 






19.15 


20.13 


20.24 


21.41 






18.93 


19.81 


19.56 


19.77 




Means 


18.1 


19.2 


19.2 


19.4 


19.0 



I 





20.35 


18.83 


17.43 


19.43 






19.88 


19.43 


20.38 


18.40 






20.72 


10.45 


18.92 


21.09 






18.33 


20.23 


19.90 


21.70 




Written 


18.97 


17.00 


20.89 


20.86 






17.38 


19.67 


20.91 


17.27 




t 


18.52 


19.66 


19.73 


16.76 






17.50 


18.69 


19.90 


18.72 






21.55 


20.99 


19.00 


17.74 






20.21 


15.94 


19.93 


20.91 




Means 


19.3 


18.1 


19.7 


19.3 


18.9 


Means for 












Treatments 


18.71 


18.65 


18.97 


19.35 


19.0 










(Grand Means) 
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APPENDIX E (CONTINUED) 



TABLE 18 

MEANS FOR STUDENT-TEACHER RATINGS 
FIFTH CYCLE 



Feedback 


1 


TREATMENTS 
2 3 


Means for 
4 Total 




19.34 


17.41 


17.59 


14.96 






18.93 


20.44 


15.76 


17.65 






18.93 


17.17 


18.59 


20.19 






19.33 


17.20 


21.19 


20.38 




Face-to-Face 


15.17 


21.58 


22.57 


21.36 






19.42 


18.57 


20.04 


20.65 






15.43 


19.31 


21.66 


16.18 






20.04 


21.54 


16.69 


19.75 






19.96 


22.88 


20.64 


18.29 






17.41 


18.75 


19.48 


19.30 




Means 


18.4 


19.5 


19.4 


18.9 


19.0 


21.66 


17.88 


19.61 


18.72 






22.33 


20.16 


22.30 


19.69 




i 


23.96 


19.70 


20.69 


20.79 






20.39 


22.26 


20.55 


21.90 




Written 


19.88 


15.00 


21.18 


21.12 




t 


20.92 


19.43 


21.24 


17.66 






18.70 


22.50 


21.62 


19.33 




1 


18.16 


18.06 


19.34 


19.00 






22.19 


21.00 


16.20 


15.25’ 






20.78 


14.71 


20.32 


22.14 




Means 

Means for 
Treatments 


20.8 , 


19.1 


20.3 


19.6 


19.9 


19.62 


19.28 


19.86 


19.22 19.5 

(Grand -Means)__ 
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APPENDIX E (CONTINUED) 



TABLE 19 

ANALYSIS OF VARIANCE FOR HAYES REACTION SCALE 

CYCLE 1 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


246.23 




11f 


Treatments 


3 


10.91 


3.64 


1.191 


Type Feedback 


1 


.32 


.32 


.105 


Interaction 


3 


15.29 


5.10 


1.670 


Within (error) 


72 


219.71 


3.05 





TABLE 20 

ANALYSIS OF VARIANCE FOR HAYES REACTION SCALE 

CYCLE 5 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


351.56 




IM* 


Treatments 


3 


5.67 


1.89 


.443 


Type Feedback 


1 


16.74 


16.74 


3.919 


Interaction 


3 


21.68 


7.23 


1.692 


Within (error) 


72 


307.47 


4.27 





APPENDIX E (CONTINUED) 



TABLE 21 

COEFFICIENTS OF REPRODUCIBILITY 
HAYES PUPIL-TEACHER REACTION SCALE 



Item 


First Rating 


Second Rating 


Final Rating 


1 


.83 


.81 


.82 


2 


.78 


.82 


.83 


3 


.79 


.83 


.83 


4 


.78 


.86 


.87 


5 


.76 


.80 


.81 


6 


.77 


.80 


.80 


7 


.78 


.84 


.84 


8 


.75 


.74 


.75 


9 


.75 


.80 


.79 


Average 


.78 


.81 


.82 





HAYES PUPIL- 


TABLE 22 
MARGINALS 

-TEACHER REACTION SCALE 




Item 


First Rating 


Second Rating 


Final Rating 


' 1 


.34 


.30 


.30 


2 


.74 


.79 


.77 


3 


.24 


.26 


.26 


4 


.74 


.79 


.80 


5 


.38 


.44 


.40 


6 


.34 


.40 


.39 


7 


.63 


.70 


.70 


8 


.29 


.34 


.36 


9 


.58 


.57 


.56 


Average 


.47 


.51 


.50 
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APPENDIX E (CONTINUED) 



TABLE 23 

■INTENSITY ANALYSIS 
FOR CYCLES 1, 2, 3, AND 4 

HAYES PUPIL-TEACHER REACTION SCALE 
CONTENT SCORE 



Intensity 

Score 


0 




2 


3 


4 


5 


6 


7 


8 


9 


Total ! 


Cum 
7 0 tile 


9 


1 


0 


0 


0 


0 


0 


0 


1 


2 


154 


158 


100 


8 


1 


0 


0 


0 


1 


0 


1 


6 


97 


76 


182 


98 


7 


1 


2 


1 


1 


0 


3 


2 


96 


128 


42 


276 


95 


6 


2 


34 


5. 


1 


4 


9 


79 


210 


139 


12 


495 


?JL 


5 


4 


11 


4 


7 


17 


103 


269 


280 


42 


0 


737 


83 


4 


8 


15 


26 


35 


132 


334 


363 


115 


0 


0 


1028 


71 


3 


14 


25 


61 


158 


309 


399 


130 


0 


0 


0 


1096 


55 


2 


18 


58 


188 


307 


399 


137 


1 


0 


0 


0 


1108 


38 


1 


41 


156 


272 


252 


105 


0 


0 


0 


0 


0 


826 


21 


0 


98 


196 


146 


112 


0 


0 


0 


0 


0 


0 


552 


9 


Totals 


188 


497 


703 


873 


967 


985 


845 


708 


408 


284 


6458 




Cum 

% tiles 


3 


11 


21 


35 


50 


65 


78 


89 


96 


100 






Midpoint: 

of 

Content 
% tiles 


1.5 


7 


16.5 


28 


42.5 


57.5 


71.5 


83.5 


92.5 


98 






Median of 
Intensity 
% tiles 


8.6 


13.0 


,18 « 1 


25.0 


37.1 


,53.1 


67.9 


81.2 


,90.9 


98.2 
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APPENDIX E (CONTINUED) 

TABLE 24 

INTENSITY ANALYSIS 
FOR CYCLES 5 AND 6 

HAYES PUPIL-TEACHER REACTION SCALE 
CONTENT SCORE 



Intensity 

Score 


k # 

0 


1 


2 


3 


4 


5 


6 


j 

7 i 


! 8 


9 


Total 


Cum 
% tile 


9 


9 


1 


0 


0 


0 


0 


1 


| i 

1 


1 

6 


94 


103 


100 


8 


1 


0 


0 


0 


0 


0 


2 


5 


79 


36 


123 


96 


7 


0 


1 


0 


1 


0 


2 


3 


58 


87 


22 


174 


92 


6 


1 


1 


1 


1 


4 


6 


38 


69 


62 


3 


186 


86 


5 


4 


6 


3‘ 


3 


8 


48 


115 


106 


22 


0 


315 


80 


4 


2 


9 


7 


16 


40 


125 


136 


35 


0 


0 


3/0 


69 


3 


6 


16 


19 


52 


113 


176 


60 


0 


0 


0 


442 


56 


2 


13 


24 


79 


124 


170 


65 


0 


0 


0 


0 


475 


41 


1 


20 


88 


113 


127 


67 


0 


0 


0 


0 


0 


415 


25 


0 


62 


91 


88 


74 


0 


0 


0 


0 


0 


3 


318 


11 


Totals 


109 


237 


310 


398 


402 


422 


355 


274 


256 


158 


2921 




Cum 

% tiles 


4 


12 


22 


36 


50 


64 


76 


86 


95 


, 100 






Midpoint 

of 

Content 
% tiles 


2 


8 


17 


29 


43 


57 


70 


i 

81 


90.5 


97.5 






Median of 
Intensity 
% tiles 


9.67 , 


15.4 


19.3 i 


i 24.8 


37.6 


53.4 


67.2 


79.6* 


•89.0 , 


96.8 
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istnents 15.09 17.58 20.84 24.97 31.05 17.42 25.57 16.86 23.14 19.21 

(Grand Means) 



. APPENDIX E (CONTINUED) 



TABLE 27 

ANALYSIS OF VARIANCE LARGE I/D 
CYCLE 1 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


1182.59 






Treatments 


3 


26.18 


8.73 


.572 


Type Feedback 


1 


40.86 


40.86 


2.68 


Interaction 


3 


17.80 


5.93 


.389 


Within (error) 


72 


1097.75 


15.25 


— 





ANALYSIS 


TABLE 28 
OF VARIANCE 
CYCLE 5 


LARGE I/D 




Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


5314.36 






Treatments 


3 


82.87 ' 


27~62 


7399 


Type Feedback 


1 


110.80 


110.80 


1.602 


Interaction 


3 


141.79 


47.26 


.683 


Within (error) 


• 72 


4978.89 


69.15 
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APPENDIX E (CONTINUED) 



TABLE 29 





ANALYSIS 


OF VARIANCE 
CYCLE 1 


REVISED i/d 






Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 



Total 


79 


62024.02 






Treatments 


3 


2771.02 


923.67 


1.156 


Type Feedback 


1 


1330.65 


1330.65 


1.665 


Interaction 


3 


403.25 


134.42 


.168 


Within (error) 


72 


57519.10 


798.88 


- 



TABLE 30 





ANALYSIS 


OF VARIANCE 
CYCLE 5 


REVISED i/d 






Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 



Total 


79 


36375.91 


— 


- 


Treatments 


3 


889.78 


296.59 


.636 


Type Feedback 


1 


961.54 


961.54 


2.063 


Interaction 


3 


967.83 


322.61 


.692 


Within (error) 


72 


33556.77 


466.07 
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APPENDIX E (CONTINUED) 



TABLE 31 

INTERACTION ANALYSIS RESULTS 
CYCLES 1, 2, 3 AND 4 







Range of 


Scores 


Percentage of Time 




Categories 


Highest 


Lowest 


Average Score 


1 . 


Accepting student 
feeling 


2.27 


0 


.15 


2. 


Praising students 


9.77 


0 


3.10 


3. 


Accepting or using 
student ideas 


26.54 


1.22 


15.22 


4. 


Asking questions for 
students to answer 


41.49 


5.17 


23.19 


5. 


Lecturing 


48.68 


.81 


15.40 


6. 


Giving directions 


7.31 


0 


1.44 


7. 


Criticizing students 
or justifying 
teacher authority 


3.22 


0 


.55 


8. 

\ 


Talk by students in 
response to teacher 


60.88 


4.60 


20.71 


9. 


Talk initiated by 
students * 


60.59 


0 


16.92 


10. 


Periods of silence or 
confusion 


26.72 


.21 


3.22 
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APPENDIX E (CONTINUED) 



TABLE 32 
MEANS FOR MTAI 1 
PRE-TEST 



Feedback 




TREATMENTS 




Means for 
Total 


1 


2 


3 


4 




32 


44 


82 


90 






76 


108 


84 


95 






21 


72 


79 


83 






61 


64 


65 


120 




Face-to-Face 


81 


87 


95 


105 






64 


50 


63 


85 






134 


83 


76 


110 






100 


57 


70 


109 






58 


0 


56 


87 






114 


76 


46 


39 




Means 


74.1 


64.1 


71.6 


92.3 


75.5 




44 


70 


72 


82 






52 


76 


58 


56 






70 


41 


52 


73 






110 


51 


52 


109 




Written 


54 


66 


56 


62 






100 


87 


94 


80 






51 


38 


82 


53 






42 


33 


65 


67 




* 


103 


89 


89 


104 




• 


30 


73 


62 


81 




Means 


65 . 6 


62.4 


68.2 


76.7 


68.2 


Means for 


« 










Treatments 


69.9 


63.3 


69.9 


84.5 


71.9 












(Grand Means) 



l*To eliminate negative MTAI means, a constant of 27 was added 
to each score. 
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APPENDIX E (CONTINUED) 



TABLE 33 
MEANS FOR MTAll 
POST-TEST 



Feedback 


i 


TREATMENTS 
2 . 3 


4 


Means for 
Total 




23 


22 


91 


70 






107 


125 


85 


95 






4 


97 


82 


98 




Face-to-Face 


50 


34 


IB 


154 






62 


126 


139 


92 






'85 


16 


107 


64 






144 


82 


84 


109 






89 


79 


114 


116 






64 


0 


68 


90 






142 


85 


31 


88 




Means 


77.0 


66.6 


87.6 


97.6 


82.2 




19 


74 


56 


94 






119 


61 


65 


46 






84 


35 


78 


71 






133 


71 


55 


137 




Written 


43 


64 


32 


77 






115 


108 


71 


91 






69 


35 


101 


90 






95 


59 


47 


96 




t 


78 


113 


100 


117 






52 


89 


74 


93 




Means 


. 80.7 


70.9 


67.9 


91.2 


77.7 


Means for 
Treatments 


78.9 


68.8 


77.8 


94.4 79.9 

(Grand Means) 



^To eliminate negative MTAI means, a constant of 33 was added 
to each score. 



E-19 



o 



APPENDIX E (CONTINUED) 



. TABLE 34 

ANALYSIS OF VARIANCE MTAI (PRE-TEST) 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


48146.75 






Treatments 


3 


4835:65 


1611.88 


2.79* 


Feedback Level 


1 


1065.80 


1065.80 


1.84 


Interaction 


3 


584.50 


194.83 


.34 


Within (error) 


72 


41660.80 


578.62 


— 



*2.74 needed at .05 level to be significant 



TABLE 35 

ANALYSIS OF VARIANCE MTAI (POST-TEST) 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


79 


88104.69 






Treatments 


3 


6805.84 


2268.61 


2.07 


Feedback Level 


1 


409.52 


409.52 


.37 


Interaction 


• 3 


1896.63 


632.21 


.58 


Within (error) 


72 


78992.70 


1097.12 


- 



* 



APPENDIX E (CONTINUED) 



TABLE 36 
MEANS FOR MTAI 
ANALYSIS OF COVARIANCE 



Source of 
Variation 


df 


x 2 


xy 


y 2 


Total 


78 


48146.75 


50660.37 


88104.69 


Treatments 


3 


4835.65 


5712.07 


6805.84 


Within (error) 


75 


43311.30 


44948.30 


81298.85 



Source of 
Variation 


df 


Sum of * 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


78 


34799.47 


446.15 


mm 


Treatments 


3 


148.01 


49.34 


.11 


Within (error) 


75 


34651.46 


462.02 


1.04 



APPENDIX E (CONTINUED) 



TABLE 37 

COEFFICIENTS OF CONSISTENCY 
HAYES PUPIL-TEACHER REACTION SCALE 



TREATMENT 1 


Cycles 


r_ 


Number of Weeks 
Between Ratings 


1 & 2 


.77 


‘3 


1 & 3 


.75 


6 


1 & 4 


.69 


9 


1 & 5 


.74 


21 


1 & 6 


.70 


25 


2 & 3 


.81 


3 


2 & 4 


.85 


6 


3 & 4 


.85 


3 


5 & 6 


.79 


3 


t 


TREATMENTS 1, 2, 3, AND 4 


Cycles 


■r 


Number of Weeks 
Between Ratings 


1 & 2 


.73 


3 



0 



1 & 5 



58 



21 
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•«* Significant at the .01 level. 

* Significant at the .05 level. 

K=80 for all of the above correlations 



APPENDIX E (CONTINUED) 



TABLE 39. 

ANALYSIS OF VARIANCE OF 
PRE-STANFORD STUDENT ACHIEVEMENT SCORES 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


1911 


1079428.80 




MM 


Treatments 


3 


'3745.40 


1248.47 


2.216 


Type Feedback 


1 


3.00 


3.00 


.005 


Interaction 


3 


2765.10 


921.70 


1.636 


Within (Error) 


1504 


1072915.30 


563.51 





TABLE 40 

ANALYSIS OF VARIANCE OF 
POST-STANFORD STUDENT ACHIEVEMENT SCORES 


Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


1911 


1174742.80 


MM 


M* 


Treatments 


3 


4360.80 


1453.60 


2.369 


Type Feedback 


1 


1920.20 


1920.20 


3.130 


Interaction 


3 


297.60 


99.20 


0.162 


Within (Error) 

# 


1904 


1168164.20 


613.53 


— 



APPENDIX E (CONTINUED) 



TABLE 41 

STANFORD STUDENT ACHIEVEMENT SCORES 
ANALYSIS OF COVARIANCE 



Source of 
Variation 


df 


x2 


xy 


y 2 


Total 


1910 


1079428.50 


1035850.00 


1174742.30 


Treatments 


3 


3745.30 


3719.90 


4360.50 


Type of Feedback 


1 


2.80 


73.30 


1919.80 


Interaction 


3 


2765.20 


821.40 


297.80 


Within (error) 


1903 


1072915.20 


1031235.40 


1168164.20 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


1910 


180710.83 




mm 


Treatments 


3 


669,67 


223.22 


2.400 


Type of Feedback 


1 


1781.48 


1781.48 


19.155** 


Interaction 


3 


1270.22 


423.41 


4.552** 


Within (error) 


1903 


176989.46 


93.01 


- 



** Significant at .01 level 



ADJUSTED STUDENT ACHIEVEMENT MEANS 



Treatment 1 64.88 
Treatment 2 63.29 
Treatment 3 63.62 
Treatment 4- 63.95 
Face-to-Face Feedback 62.97 
Written Feedback 64.90 
Face-to-Face Feedback, Treatment 1 62.93 
Written Feedback, Treatment 1 66.83 
Face-to-Face Feedback, Treatment 2 61.77 
Written Feedback, Treatment 2 64.82 
Face-to-Face Feedback, Treatment 3 63.11 
Written Feedback, Treatment 3 64.14 
Face-to-Face Feedback, Treatment 4 64*08 
Written Feedback, Treatment 4 63.83 



APPENDIX E (CONTINUED) 



TABLE 42 

ANALYSIS OF VARIANCE OF 



Source of 




Sum of 


• 

Variance 


F 


Variation 


df 


Squares 


Estimate 


Ratio 


Total 


1911 


227537.00 


MM 


MM 


Treatments 


3 


1304.00 


434.67 


3.669* 


Type Feedback 


1 


239.00 


239.00 


2.018 


Interaction 


3 


451.00 


150.33 


1.269 


Within (error) 


1904 


225543.00 


118.46 




^Significant at 


.05 level 










TABLE 43 








ANALYSIS 


OF VARIANCE OF 






STUDENT ATTITUDES TOWARD 


SCHOOL SUBJECTS 


(POST-TEST) 




Source of 




Sum of 


Variance 


F 


Variation 


df 


Squares 


Estimate 


Patio 


Total 


1911 


217700.00 




«*• 


Treatments 


3 


121.00 


40.33 


.354 


Type Feedback 


1 


3.00 


3.00 


.026 


interaction 


3 


509.00 


169.67 


1.488 


Within (error) 


1904 


217067.00 


114.01 





E-26 




APPENDIX E (CONTINUED) 



TABLE 44 

STUDENT ATTITUDE TOWARD SCHOOL SUBJECTS 
ANALYSIS OF COVARIANCE 



Source of 
Variation 


df 




xy 


y 2 


Total 


1910 


227537.00 


80102.00 


217699.00 


Treatments 


3 


1303.00 


188.00 


120.00 


Type of Feedback 


1 


239.00 


24.00 


2.00 


Interaction 


3 


453.00 


234.00 


509.00 


Within (error) 


1903 


225542.00 


79656.00 


217068.00 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


1910 


189499.86 






Treatments 


3 


149.41 


49.80 


0.502 


Type of Feedback 


1 


14.84 


14.84 


0.149 


Interaction 


3 


400.19 


133.40 


1.344 


Within (error) 


1903 


188935.42 


99.28 





APPENDIX’ E (CONTINUED) 

TABLE 45 

ANALYSIS OF VARIANCE FOR HAYES REACTION SCALE 

CYCLE 1 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


1911 


>7693.93 


mm 




Treatments 


3 


13.05 


4.35 


1.084 


Type Feedback 


1 


12.08 


12.08 


3.012 


Interaction 


3 


29.52 


9.84 


2.452 


Within (error) 


1904 


7639.27 


4.01 


- 



TABLE 46 

ANALYSIS OF VARIANCE FOR HAYES REACTION SCALE 

CYCLE 5 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Tota.1 


1911 


10320.61 




mm 


Treatments 


3 


28.95 


9.65 


1.823 


Type Feedback 


1 


109.71 


109.71 


20.725** 


Interaction 


3 


102.78 


34.26 


6.472** 


Within (error) 


1904 


10079.17 


5.29 


mm 


** Significant at 


.01 level 









E-28 



o 

ERLC 



APPENDIX E (CONTINUED) 



TABLE. 47 



ANALYSIS OF COVARIANCE FOR HAYES REACTION SCALE 



Source of 
Variation 


df 






y 2 


Total 


1910 


7693.93 


4045.16 


10320.61 


Treatments 


3 


13.05 


-5.16 


28.94 


Type Feedback 


1 


12.08 


36.41 


109.71 


Interaction 


3 


29.52 


45.68 


102.79 


Within (error) 


1903 


7639.27 


3968.24 


10079.17 



Source of 
Variation 


df 


Sum of 
Squares 


Variance 

Estimate 


F 

Ratio 


Total 


1910 


8193.87 


— 




Treatments 


3 


37.81 


12.60 


2.991* 


Type of Feedback 1 


75.02 


75.02 


17.807** 


Interaction 


3 


63.18 


21.06 


4.998** 


Within (error) 


1903 


8017.86 


4.21 


- 


* Significant 
** Significant 


at . 05 level 
at . 01 level 









ADJUSTED STUDENT ACHIEVEMENT MEANS 


Treatment 1 




4.70 


Treatment 2 




4.46 


Treatment 3 




4.71 


Treatment 4 




4.40 


Face-to-Face Feedback 




4.37 


. Written Feedback 




4.77 


Face-to-Face Feedback, Treatment 


1 


4.25 


Written Feedback, Treatment 1 




5.16 


Face-to-Face Feedback, Treatment 


2 


4.52 


Written Feedback, Treatment 2 




4.40 


Face-to-Face Feedback, Treatment 


3 


4.54 


Written Feedback, Treatment 3 




4.89 


Face-to-Face Feedback, Treatment 


4 


4.18 


Written Feedback, Treatment 4 




4.62 



E-29 
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